This week was the annual 3 day Biomed/MIXiii (I have no idea what MIXiii means btw) conference in Tel Aviv. The organizers also billed it as the “18th National Life Science and Technology Week” (which I also do not know what that means). This was a particular difficult time for a conference of medical device and pharma in Tel Aviv since it coincided with the Eurovision 2019 activities – and the traffic was tough.
There were a huge number of lectures and participants from all over the world and I suppose from that perspective, the conference is a success and tribute to the burgeoning Israeli biomed industry. Forbes calls Biomed “The High-Paying, Low-Stress STEM Job You Probably Haven’t Considered”. I think that this is probably a good description for the conference – high participation but low stress.
My colleagues and I come to the conference to network, schmooze, meet customers and suppliers. It’s a good opportunity to take a few meetings, say hi to friends and hustle for new business. Having said that, I did meet a few really interesting companies:
RCRI – is a Minneapolis MN based medical device CRO. I met Todd Anderson and his boss Lisa Olson and pitched our approach for fast data in clinical trials to assure high levels of patient compliance to the protocol and submit faster to FDA. Todd and Lisa get it and they were open about the CRO business model being more people-hours not speed. They seemed genuinely interested in what we are working on but its hard to tell with Americans.
Docdok Health – is a startup founded by Yves Nordman, who is a Swiss MD living in Carmiel. It’s a doctor-patient communications platform beginning to branch out into Post-marketing studies with RWD. We shared demos and it seems that there is synergy between our regulatory platform and their post-marketing work.
Resbiomed – met Alex Angelov, the CEO. Alex is leading a consortium including Flaskdata, Carl Zeiss, Collplant, PreciseBio and Pluristem for a Horizon2020 submission for an amazing project for an implant to the cornea. Dan Peres from Pluristem got us together. Cheer for us!
BSP Medical and ICB (Israel China Biotech investment) – my buddy Hadas Kligman literally took me by hand to visit to Yehuda Bruner and Andrew Zhang and I did my 60s elevator pitch on getting medical device companies to FDA/CFDA 6-12 months faster. We agree to talk after the conference.
Butterfly Medical – I met Idan Geva, the CEO last year at Biomed – we ate lunch at the same table. I pitched him but he was uninterested – they were using EDC2Go – and he didn’t want to hear other options. At the Minnesota pavilion talking to Todd Anderson from RCRI, Idan shows up and looks at me and says “Heah – Hi Danny – I left a contact me request on your web site yesterday and no one got back to me. I said shame on us. He says – he was referred to us by someone from Florida who used to use Medidata. I asked where/who? was it Miami? He says yeah it was Miami and checks his phone – says its someone from Precision Clinical Research that are using Flaskdata and recommended. (Precision is one of our customer’s Miami sites). I asked what happened to EDC2Go – he said well you know – they are end of life (I think this means the end of low-cost EDC) and we are now entering questionnaires manually on paper and it is driving us crazy. He said – can you stick around and give us a demo at 15:00? I said sure. We met at 15:00 by the bar upstairs in the David Intercontinental and I demoed the system – he said “Show me the Forms designer”. I showed him. He says “show me how CRC enters data” – I showed him. He says “Show me how to extract data” – I showed him. I think he actually did not believe how fast the Extract to CSV process was and asked me twice if that was the data. In the end – the format of Mac Numbers was a bit strange for him. I showed him a quick presentation – and he saw that Serenno is a customer – and says – “Heah Tomer is a neighbor of ours in the incubator in Yokneam”. He asked how much and I said $2K for a basic onboarding package and $1500 / month. Or $10K and we will build the CRF (their CRF is super simple btw). He wanted a discount, being Israeli. I said – “lets meet with your clinical person and get her to buy-in to the solution. If she buys in – you and I can talk business but before that, there is no point horse-trading.
Count the probabilities of this happening and you will see that it is an impossible event.
Thursday I went back to demo Todd and meet Dr Yael Hayun from Syqe Medical. Yael is one of the most impressive people I’ve met in a long time. She is an MD from Hadassah and one of the movers and shakers in LogicBio Therapeutics. After we chatted – I told her that Syqe is lucky to have her onboard. I did our Today is about Speed presentation and a short demo. She was suitably impressed and then mentioned they had met with a Danish EDC company called Smart Trial – which turns out is yet another low-cost eCRF provider. I said look – eCRF is like 10% of the solution you need – in the case of Syqe, you have a digital inhaler and with cannabis, you are going to have a lot of concerns about patient compliance.
This is what we do – fast data collection from patients, investigators and digital inhalers and automated deviation detection and response.
On the way back – huge traffic from Eurovision. Didn’t hear a single lecture but the meetings and people were outstanding.
Living off generic solutions developed in the past
I recently read some posts on Fred Wilson’s blog and it was impressive that he writes every day.
I’ve fallen into the trap of collecting raw material and then waiting to find time to write a 2000-word essay on some topic of importance to me. But, I think it was Steve Jobs who said the best time to do anything was 20 years ago and failing that – best time is now. So now – I will start writing every day and attempt to write on topics of interest to my customers, not me.
We are working on automating patient compliance in medical device clinical trials. Patient compliance is critical for the success of medical device studies.
When we mean success – we mean proving or disproving the scientific hypothesis of the study. Efficacy – is the device an effective treatment for the indication?
Safety – is the device safe for patients?
When we say patient compliance automation we mean the combination of 4 things which depend on each other:
1.Reinforcing patient compliance to the protocol – for example reporting on time and taking the treatment on time. AI-based reinforcement uses data from the patient’s behavior and similar behavior to keep the patient on track without driving him crazy with text or push messaging.
2.Automated monitoring of compliance – using clinical measures and the treatment schedule for the study. An example of a clinical measure is the number of capsules a patient took. An example of treatment schedule is taking the capsules every day before 12.
The output of automated monitoring is real-time alerts and compliance trends to the study team.
3. Automate patient compliance reinforcement using and adaptive control process that takes fresh data from the alerts to make decisions on how to reinforce the patient and keep them on track.
4.In order to automate monitoring and do AI-based reinforcement of patient compliance, you need fresh and up-to-date data.
There is a lot of work being done by startups like Medable, Litmus Health and Flaskdata.io (disclaimer – I am the founder of Flaskdata.io) but it’s a drop in the ocean of 24,000 new clinical trials every year.
Fundamentally – the problem is that the clinical trials industry uses generic solutions developed 40 years ago to assure quality of data-entry from paper forms.
The generic solution used today involves waiting 1-3 days for site data collection to the EDC, and 4-6 weeks for a site visit and SDV and then another 1-12 weeks for a central monitoring operation in your CRO to decide that there was a protocol violation.
You don’t have to be a PhD data scientist to understand that you cannot assure patient compliance to the clinical protocol with 12-week-old data.
The only explanation for using 40-year-old generic solutions is that the CRO business model is based on maximizing billable hours instead of maximizing patient compliance.
It seems that if you want to achieve real-time detection and response and AI-based patient compliance reinforcement, you have to disrupt the CRO business model first.
The perverse incentive for the high costs of medical devices and delay to market
The CRO outsourcing model and high US hospital prices result in higher total CRO profits via higher costs to companies developing innovative medical devices. These costs are passed down to consumers after FDA clearance.
We’ll take a look at the cost dynamics of medical device clinical trials and the clinical trial value chain.
We’ll then consider an alternative business model that changes the way medical device sponsors conduct clinical trials, reduce their costs by 70-80% and shortens time to FDA submission.
The high costs of US hospitals
By 2000, the US spent more on healthcare than any other country, whether measured per capita or a percentage of GDP.
U.S. per capita health spending was $4,631 in 2000, an increase of 6.3 percent over 1999. 4 The U.S. level was 44 percent higher than Switzerland’s, the country with the next-highest expenditure per capita; 83 percent higher than neighboring Canada; and 134 percent higher than the OECD median of $1,983. 5
In 2011, the US Affordable Care Act set a requirement for MLR (Medical Loss Ratio) that insurers must spend 80-85% of revenue on medical services. This reduced insurer margins, and drove up hospital prices to make up for lower margin.
The CRO business model
CROS (clinical research organizations) are outsourcing businesses that provide an array of services for clinical trial management and monitoring, reporting and regulatory submission. For medical device studies, CROS employ 2 basic outsourcing models, people sourcing and functional sourcing. In people out-sourcing, the medical device company is responsible for managing contractors. In functional outsourcing, the company may buy a set of functions, for example study monitoring and medical writing.
Neither CRO model has an explicit incentive to complete a study faster since that would reduce outsourcing revenue for the CRO. The more time a CRO spends on monitoring, site visits, SDV and study closeout, the more revenue it generates.
A medical device sponsor may elect to do it himself which shifts the CRO cost to an internal headcount cost supplemented with additional costs for consultants with risk and time delays by not having the CRO expertise and infrastructure. There is tacitly no free lunch, as we will discuss later in this article.
The result is a perverse incentive for delay and higher costs to bring innovative medical devices to market.
The CRO business model combined with higher hospital prices drive higher total profits via higher costs to customers. The higher cost of innovative medical devices is then passed down to consumers (patients) after FDA clearance.
Consumer value chains
A consumer value chain looks generically like this:
Suppliers -> Distributers -> Consumers
By the early 90’s, the PC industry led by Intel and Microsoft used a 2-tier value chain:
Resellers were further segmented according the customer size and industry segment – Retail, Large accounts, SMB and VARS (value-added-resellers) selling their own products and services to a particular industry vertical. The PC industry value-chain model left Microsoft with 50% of the SRP (suggested retail price) and delivered products to customers that were 45-50% less than SRP, leaving the channel with 0-5%.
The channel was forced to implement extremely efficient operations and systems and sell value-added services and products in order to survive.
By the new millennium, Apple introduced a 1-tier model with a user-experience designed and controlled by Apple.
The Apple 1-tier channel looks like this:
Eventually the Apple channel model broadened to include a 2-tier model similar to PC industry:
By the mid-2000s, Amazon AWS (and generally the entire cloud service / SaaS industry) evolved the channel model to 0-tiers with a direct subscription and delivery model.
As AWS grew and introduced spot pricing, an aggregation sub-market developed, looking extremely similar to movie and TV distribution models.
AWS also became a distribution channel for other cloud products similar to content distribution (Think Netflix).
Outstanding user-experience and aggregation are the hallmarks of companies like Airbnb, Netflix and Uber.
The common thread is that AWS and Netflix deliver a digital product end-to-end, whereas Airbnb and Uber aggregate trusted suppliers inside the Airbnb and Uber brand environment and provide an outstanding and uniform user experience to all the consumers. This is in contrast to the variegate user experience a customer got from the 90’s Microsoft channel. There are great resellers and terrible resellers.
We will return to user experience and aggregation later.
Little has changed in the past 71 years regarding clinical trials. Clinical trials and hospital operations now have a plethora of complex expensive, difficult-to-use IT with a value chain that provides a dystopian user experience for hospitals, patients and medical device companies.
HCOs (healthcare operators) rely on data collection technology procured by companies running clinical research (sponsors and CROs). This creates a number of inefficiencies:
1 – HCO staff are faced with a variety of systems on a study by study basis. This results in a large amount of time spent learning new systems, staff frustration and increased mistakes. This is passed on in costs and time to sponsors after CRO markup.
2 – The industry is trending towards the use of eSource and EMR to EDC data transfer. eSource/ePRO tools need to be integrated into the patient care process. Integration of EMR with EDC becomes logistically difficult due to the number of EDC vendors on the market (around 50 established companies).
3 – Siloed data collection in hospitals with subsequent manual data re-entry results in large monitoring budgets for Source Data Verification, and delays caused by data entry errors and related query resolution. Delays can be on the order of weeks and months.
4 – Use of multiple disconnected clinical systems in the hospital creates a threat surface of vendor risk, interface vulnerabilities and regulatory exposure.
Losing focus on patients
One of the consequences of the 3-tier medical device value chain is loss of focus on the patient user experience. Upstream and to the left, patients are ‘subjects’ of the trial. The patient reported outcomes apps they use vary from study to study. Downstream and to the right (what FDA calls ‘post-marketing’), patients are consumers of the medical device and the real-world user experience is totally different than the UX in the study. The real-world data of device efficacy and safety is disconnected from the clinical trial data of device efficacy and safety.
Clinical trial validation
Patient compliance is critical to clinical trial validation of medical device. Who owns patient compliance to the research protocol? The medical device sponsor, the CRO, the hospital site or the subject? The CRO may not collect a patient compliance metric since he outsources to the hospital. The hospital may not have the tools and the medical device company is outside the loop. My essay on determining when patient compliance is important in medical device trials goes into more detail on the problem of losing focus on the patient.
Vertical integration and aggregation
We previously made a qualitative claim that hospital site costs are high for medical device studies. How high are they relative to consumer healthcare?
In a medical device trial recently done on the Flaskdata.io platform, the sponsor paid the hospital investigatory sites $700K for a 100 subject, 7 month multi-site study. (There were no medical imaging and blood test requirements).
In 2016, Medicare Advantage primary care spend was $83 PMPM (per member per month). Let’s say that a premium service should cost $100 PMPM. Let’s use that as a benchmark for the cost of processing a patient in a medical device trial. Take this medical device Phase II medical device trial with 100 patients, running for 7 months:
That’s 100 x 7 x 100 = $70K for patients. Not $700K.
Perhaps the law of small numbers is killing us here. The way to solve that is with aggregation and vertical integration. Let’s return to the medical device clinical trial value chain. As we can see, there are too many moving parts and a disconnect between patients in the clinical studies and consumers in the real world.
One alternative is to integrate backwards and to the left. This requires managing hospital site functions and to a certain degree is done in the SMO (site management organization model).
The other alternative is to integrate forward and to the right. This is the path that Airbnb, Uber and Netflix took aggregating consumer demand with an outstanding user experience. The aggregation gives Airbnb, Uber and Netflix buying power to the left, enabling them to choose the best and most cost-effective suppliers.
The value chain would then look like this:
Suppliers->Medical device companies->Patients
This is a model that we see increasingly with Israeli medical device vendors with limited budgets. The Medical device company uses a cloud platform to collect digital feeds from investigators, patients and devices and automate monitoring for deviations. Focus on the patient user experience begins with design of the device and continues to post-marketing. Aggregation of patients enables purchasing power with suppliers – research sites, clinical consultants and study monitors.
Short-term versus long-term cost allocation
The reality is that using a technology platform for vertical integration is more expensive initially for implementation by the medical device company. It should be.
Under-funding your infrastructure results in time delays and cost spikes to the medical device sponsor at the end of the study.
The current CRO methodology of study close-out at the end of a clinical trial lowers costs during the trial but creates an expensive catch-up process at the end of the study.
The catch-up process of identifying and closing discrepancies can take 2-6 months depending on the size and number of sites. The catch-up process is expensive, delaying submission to FDA and revenue since you have to deal with messy datasets. The rule of thumb is that it costs 100X more to fix a defect after the product is manufactured than during the manufacturing process. This is true for clinical trials as well. A real-time alert on treatment non-compliance during the study can be resolved in 5 minutes. By waiting to the end of study it will take a day of work-flow, data clarifications and emails to the PI.
Vertical integration reduces costs and delay at study-end with continuous close. It is more expensive initially for the medical device company and it should be because it accelerates time to submission and reduces monitoring and close-out costs.
The high failure rate of drugs in clinical trials, especially in the later stages of development, is a significant contributor to the costs and time associated with bringing new molecular entities to market. These costs, estimated to be in excess of $1.5 billion when capitalized over the ten to fifteen years required to develop a new chemical entity, are one of the principal drivers responsible for the ongoing retrenchment of the pharmaceutical industry. Therapeutic areas such as psychiatry, now deemed very high risk, have been widely downsized, if not abandoned entirely, by the pharmaceutical industry. The extent to which patient noncompliance has marred clinical research has in some cases been underestimated, and one step to improving the design of clinical trials may lie in better attempts to analyze patient compliance during drug testing and clinical development. Phil Skolnick, Opiant Pharmaceuticals The Secrets of a successful clinical trial, compliance, compliance, compliance.
Compliance, compliance, compliance
Compliance is considered to be key to success of a medical treatment plan. (1, 2, 3)
It is the “billion dollar question” in the pharma and medical device industry.
In home-use medical devices in particular and in chronic diseases in general – there is wide consensus that patient compliance is critical to the success of the clinical trial. Our experience with Israeli innovative medical device vendors is that they understand the criticality of patient compliance. They “get it”.
However, as Skolnick et al note – patient compliance with the clinical protocol is often underestimated in drug trials.
There are 4 challenges for assuring patient compliance in medical device trials.
1. The first challenge is maintaining transparency. An executive at IQVIA noted (in a personal conversation with me) that IQVIA does not calculate patient compliance metrics since they assume that patient compliance is the responsibility of the sites. The sponsor relies on the CRO who does not collect the metrics who relies on the sites who do not share their data.
2. The second challenge is having common standard metrics of compliance. Site performance on patient compliance may vary but if sites do not share common metrics on their patients’ compliance, the CRO and the sponsor cannot measure the most critical success factor of the study.
3. The third challenge is timely data. In the traditional clinical trial process, low-level data queries are resolved in the EDC but higher-level deviations often wait until study-closeout. The ability of a study team to properly resolve thousands of patient compliance issues months (or even years) after the patient participated is limited to say the least
4. The final and fourth challenge is what happens after the clinical trial. How do we take lessons learned from a controlled clinical trial and bring those lessons into evidence-based practice?
A general approach to measuring and sharing patient compliance metrics
A general approach to addressing these challenges should be based on standard metrics, fast data and active monitoring and reinforcement and reuse.
1. Use standard metrics for treatment and patient reporting compliance. The metrics then become a transparent indicator of performance and a tool for improvement. A simple metric of compliance might be a score based on patient reporting, treatment compliance and treatment violations. We may consider a threshold for each individual metric – for example a 3 strike rule like in baseball. A more sophisticated measure of compliance might be similar to beta in capital market theory where you measure the ‘volatility’ of individual patient compliance compared to the study as a whole. (Beta is used in the capital asset pricing model, which calculates the expected return of an asset based on its beta and expected market returns or expected study returns in our case).
2. Fast data means automating for digital data collection from patients, connected medical devices and sites eliminating paper source and SDV for the core data related to treatment and safety endpoints.
3. Actively monitor and help patients sustain a desired state of compliance to the treatment protocol, both pharmacologic and non-pharmacologic. Not everything is about pill-counting. This can be done AI-based reminders using techniques of contextual bandits and decision trees.
4. Reuse clinical trial data and extract high quality training information that can be used for evidence-based practice.
Patient compliance teardown
Measures of patient compliance can be classified into 3 broad categories:
Patient reporting – i.e how well patient reports her own outcomes
1. Treatment compliance – how well the treatment conforms to the protocol in terms of dosing quantities and times of application. 2. Research suggests that professional patients may break the pill counting model
3. Patient violations – if the patient does something contrary to the protocol like taking a rescue medication before the migraine treatment
Many heart failure patients are thought to be non-compliant with their treatment because of prior beliefs – believing that the study treatment would not help them. In the European COMET trial with over 3000 patients it was found that a Lack of belief in medication at the start of the study was a strong predictor of withdrawal from the trial (64% versus 6.8%; p < 0.0001). Those patients with very poor well-being and limited functional ability (classified as NYHA III–IV) at baseline significantly (p = 0.01) increased their belief in the regular cardiac medication but not in their study medication (4)
But numerous additional factors also contribute to patient non-compliance in clinical trials: lack of home support, cognitive decline, adverse events, depression, poor attention span, multiple concomitant medications, difficulty swallowing large pills, difficult-to-use UI in medical devices and digital therapeutics and inconveniences of urinary frequency with diuretics for heart failure patients (for example).
It seems that we can identify 6 main confounding variables that influence compliance:
1. Patient beliefs – medication is useless, or this specific medication cannot help or this particular chronic condition is un-curable
2. Concerns about side effects – this holds for investigators and for patients and may account for levels of PI non-compliance.
3. Alert fatigue – patients can be overwhelmed by too many reminder message
4. Forgetfulness – old people or young persons. Shift workers.
5. Language – the treatment instructions are in English but the patient only speaks Arabic.
6. Home support – patient lives alone or travels frequently or does not have strong support from a partner or parent for their chronic condition.
Flaskdata.io provides a HIPAA and GDPR-compliant cloud platform that unifies EDC, ePRO, eSource and connected medical devices with automated patient compliance monitoring. The latest version of Flaskdata.io provides standard compliance metrics of patient reporting and active messaging reminders to help keep patients on track. Your users can subscribe to real-time alerts and you can share metrics with the entire team.
Contact Batya for a free demo and consult and learn how fast data, metrics and active reinforcement can help you save time and money on your next study.
Danny teeters on the edge of the precipice of privacy and security. Step on the brakes not on the gas and don’t look down. Take a 500m leap of faith into the chasm of medical device clinical trials. Validate digital therapeutics. Venture into uncharted territory of medical cannabis trials.
At some stage in my “let’s do something different and risky” life after leaving the safety of Intel culture, I stumbled into cybersecurity.
Cybersecurity and privacy for medical devices
I started helping Israeli medical device and digital Health startups with privacy and security consulting. We built and analysed medical device threat models. The threat analysis approach succeeded in helping people improve their systems and privacy compliance.
Over time, the threat analysis methodology that was developed was adopted by thousands of security analysts globally – PTA Technologies.
I did not do this on my own. I owe these opportunities to my friend and colleague Mike Zeevi from Softquest Systems.
Over time, I figured out what works and how to comply with standards – HIPAA, FDA and GDPR. This came from real-life implementations and FDA submissions. I got hands-on in compliance audits with large US healthcare organisations like BC/BS Dignity Health.
Development practices for connected medical device and digital health apps
Many startups in the digital health and medical IoT space make 3 mistakes when engineering their systems.
1) First they Google. 2) Then they Guess. 3) Then they DIY when the Guesses Fail.
Some companies add an additional step: “Contract to a Software House that Talks Big” and then DIY or switch contractors.
This is a costly and risky pattern. As Jim McCarthy says –
More people have ascended bodily into heaven than have shipped great software on time.
– Jim McCarthy, Dynamics of Software Development by Jim McCarthy, Denis Gilbert
For Israeli digital health startups, there is an additional risk. This is the risk of not having an organisational memory. Youth has energy, hip viewpoints and updated expertise on latest technology. Who knew that a similar technology failed 30 years ago before you were born?
Build versus buy for digital health platforms
Digital health startups face 2 challenges. The first is an engineering challenge. The second is a validation challenge.
AWS cloud services have changed the way we engineer connected medical devices and digital health apps.
However, you need to factor in the cost and time requirements for a slew of additional activities. You need reliable DevOps, application integration, data integration, performance, configuration management, security, privacy, compliance and risk management.
The validation challenge is about clinical trials. About 4 years ago, we saw that our medical device customers wanted cheaper and faster ways to collect, monitor and analyse clinical trial data.
Building the product yourself and building a digital clinical trial systems is neither simple nor cheap. Resorting to paper studies to save money, turns short-term savings into long-term losses in time and data quality.
The solution – full-stack digital clinical trial platform
I joined forces with Jenya and we took a strange and wonderful decision to help Israeli medical device companies run clinical trials in the cloud.
This is what Flaskdata.io – patient compliance automation for medical device studies does. We provide a full-stack 21 CFR, HIPAA, GDPR compliant platform for collecting and monitoring data from investigators, patients and devices. Organisations like Theranica Therapeutics and Weizmann Institute all trust our platform for their human research. Today, Flaskdata.io helps site coordinators and clinical trial manager assure patient compliance using real-time alerts and trends at over 300 sites globally.
We work hard to bring modern technology to our customers instead of paper and save time and money.
Platform as a Service offerings like IBM Watson digital health has an amazing set of tools. You have to build your own product, integrate, test, secure, verify and validate.
By comparison, validated Software as a Service platform like Flaskdata.io enables you to get started immediately. You can design data collection using visual UI and integrate the open Flask API for medical devices. Check out our Swagger here.
There is a free tier that enables very early stage startups to start running pilots for free. And yes, we support, English, Hebrew and Chinese.
Give us a shot – you will not be sorry.
100X faster to deviation detection in medical device studies.
Patient compliance automation on the flaskdata.io platform for a medical device clinical trial is 100X faster than manual monitoring.
Automated compliance monitoring analytics and real-time alerts let you focus your site monitoring visits on work with the PI and site coordinators to take total ownership and have the right training and tools to meet their patient recruitment and patient compliance goals.
In this post, Danny Lieberman, founder of flaskdata.io , discusses when patient compliance is crucial for your medical device clinical trial and when patient compliance is a negligible factor to success of the study.
From adverse events to patient compliance
My original goal for Flaskdata.io was to use machine learning to predict onset of adverse events during interventional medical device clinical trials.
For that goal, we needed data, so we started by providing cloud EDC services for medical device clinical trials with high-touch personal service and attention to the quality of the data model. Very quickly – it become apparent that we did not have enough data (and after 20 studies, hundreds of sites and thousands of patients), we still do not have enough data to predict adverse events.
However, after performing 6 digital clinical trials in 2 chronic disease indications (acute migraine and chronic constipation) we had an epiphany – “PATIENT COMPLIANCE IS KING”.
Customers using the Flaskdata.io platform for digital clinical trials, collected data via the EDC from investigators, collected data from patients (via our ePRO app) and collected data from connected medical devices (via the Flaskdata.io medical device API). The evidence was overwhelming :
Patient compliance to the protocol is an acute issue and critical success factor to the success of a connected medical device clinical trial.
Or is it.
Who owns patient compliance? The sponsor, the CRO, the site or the subject?
This discovery of the importance of patient compliance made a profound impression on us because it came from customers and empirical data they collected in our EDC systems. This impression would not change, although we began to hear dissenting opinion on the importance of, and responsibility for patient compliance in clinical trials.
Public discussion on trends in the clinical trials industry is heavily influenced by big pharmaceutical companies, big CROs like PPD and IQVia and a $70BN/year clinical operations services industry that deal largely with oncology and biotechnology. When we spoke to biotech prospects about the ability of our digital clinical trials platform to accelerate time to regulatory submission and assure high levels of patient compliance – people smiled and said “Well automated compliance monitoring is an innovative approach, but in fact, patient compliance is not important to us”.
We then spoke with the Israel country manager of one of top 3 global CROs – and they said “Interesting question. We collect many clinical trials operations metrics, but patient compliance to the clinical protocol is not a metric we collect”. I asked – “In that case, who is in charge of patient compliance? and the answer was – the sites”. In this scheme of things, if patient compliance is not a CRO metric, then the sponsor has a blind-spot to what is possibly, the single most important factor to the success of his connected medical device clinical trial. Or not.
After that, we spoke with the country manager of one of the top 3 pharmaceutical companies and Israel and he told us again “Patient compliance is a non-issue for us. Patients come to the hospital and get treatment and there is no problem”. I asked him “What about psychiatry trials?” He replied – “well yes, everyone knows that psychiatry trials have acute issues of patient compliance”
We then went back and did the most logical thing – searching in Google for “the critical success factors of clinical trials” and there are 290 million results and a ton of empirical evidence and academic and industry research on the importance of patient adherence in clinical trials.
And this vast body of empirical data is dealing primarily with drug trials, not medical device trials. The VP Clinical of a gene therapy prospect (who had previously worked at a medical device company) told us that in gene therapy patient compliance is negligible while in medical device trials, patient compliance is acute.
Hmm again. So what does Google say?
The high failure rate of clinical trials has significant impact on providing potential curative treatments to patients in need….
One key factor that has been identified in the high failure rate of clinical trials is the adherence of patients participating in clinical trials to the dosing, treatment, and study procedures that are very carefully put in place in clinically rigorous protocols. Due to the rigor that is required in order to demonstrate an “effect” relative to a standard of care treatment, even a small deviation in medication adherence can result in a trial failing to meet its pre-specified clinical endpoint.
Additionally, the current nature of clinical studies include strict timelines and competition among sites to enroll eligible subjects which can many times result in the inclusion of subjects that are simply not “medication-compliant”. The issue of medication adherence is therefore one key factor sponsors should carefully look at monitoring closely when designing and planning the medical and operational oversight of their trials.
Unfortunately, the issue of medication adherence many times goes unmanaged and falls solely on site staff to oversee. As clinical sites are many times running multiple concurrent trials and are themselves pressed to remain productive, the one-on-one daily management of medication adherence of study subjects can many times be neglected. It is therefore in the best interest of the patients in need that sponsors look towards solutions that can help to support their clinical sites in providing additional resources to maintain close and frequent interactions with subjects enrolled in key studies. It simply is no longer sufficient to rely solely on very busy clinical practices to ensure successful adherence of patients in enrolled in trials.
I’m confused. Is compliance the best interest of the patient or the best interest of the PI, or the sponsor or all of the above? We know that the PI must monitor participants’ compliance with study requirements. Failure to monitor patients adequately can sabotage the entire study and damage the site’s reputation.
CROS not collecting patient compliance metrics. Busy sites. Lack of tools. PIs who are generally not hands-on with the patients. Sounds like a classic finger-pointing situation.
We hear of the importance of site selection, but if patient compliance is not a CRO metric, then how do we measure site performance properly?
The 4 quadrants of patient compliance
In fact, the question of HOW to measure the importance of compliance is intimately related to 4 factors – and interestingly enough is totally unrelated to the site or the PI. The 4 factors of patient compliance are:
1. How do you collect data?
2. What is the indication?
3. What does the product do?
4. How involved is the patient in the treatment?
In order to understand why there is dissenting opinion on the topic of the importance of patient compliance – we can map life-science products into 4 quadrants: (Patient-centric, Digital, Investigator-centric, Implanted). The top right quadrant in green is a digital clinical trial for chronic disease, the top left is a traditional EDC operation with varying degrees of patient involvement, the bottom left is little patient involvement and EDC data collection from paper source and the bottom right is no patient involvement but with data collection from implanted devices (an interesting and extremely important use case in its own right).
The above picture tells the whole story.
Patient compliance in clinical trials is crucial in digital clinical trials and patient-centric trials using traditional EDC and patient reported outcomes.
In the end it is about the patient – not the PI, not the site operations team and their training, policies and procedures and not about the CRO.
But hey – this is something any sponsor worth their salt already knows.
Israeli Medical device innovation for high patient compliance
One of the most challenging problems in medical device clinical trials and in real-life is how to achieve high levels of patient compliance to the protocol. Automated patient compliance technology in medical device clinical trials is confronting CROs with an unpleasant status-quo of SDV as a low-value-add, high-cost, time-consuming activity for patient compliance assurance. The approach that this company takes provides continuous patient monitoring without requiring patient compliance at all.
EarlySense is an Israeli medical device that is based on a paddle placed under the patient’s mattress that continuously monitors patient movement, HR and RR trends.
The EarlySense device helps facilitate timely interventions for patients in non-ICU settings by adding a layer of care with continuous monitoring, drawing attention to those who may show early signs of deterioration and may require clinical intervention.
Since the EarlySense contact-free sensor (it looks like a small plate) is placed under the mattress and there are no leads attached to the patient – there is no need for patient compliance.
We spoke Dalia Argaman, VP Clinical & Regulatory at EarlySense to understand how to take medical devices from the design and engineering stages, navigate regulatory pathways and execute medical device clinical trials that receive FDA approval and save lives.
“I have a BSc in Chemistry and MSc in Chemical Physics and I was lucky enough to start my career immediately after completing my education. I joined Direx (a startup employing 6-7 people at that time) and became a part of developing an innovative medical device in the field of shockwave lithotripsy which is designed to break kidney stones without invasive procedure. Beforehand the standard of care was that patients with kidney stones had to undergo surgical procedure full of discomfort and further complications. I was the one who took the medical device through clinical trials when the first prototype was created”.
It was back then that Dalia got introduced to the world of medical device clinical trials, submissions to regulatory authorities (FDA, CE, CFDA) and clinical data management.
Dalia has been on top of this innovative world through the 20 years of her career, working in several companies that develop different medical devices. Alongside success, she experienced a number of professional and personal challenges that shaped her career.
“You actually might be surprised to find out that the biggest challenge I had to overcome during my career was to try and balance family life with professional. I am a career person but I also have a family and I always had to juggle between being a mother, raising a family and moving forward on the career ladder. In addition, I think when you are a regulatory person, working with various people with different understanding, different backgrounds and fields and trying to get everyone at the same page is a huge professional challenge”, she said.
While working for Glucon (Developer of Non-Invasive Glucose Monitoring Devices in Israel) Dalia was involved in clinical trials with patients who had diabetes: that included children as well. She recalls that this experience was most memorable through her entire career as it required creating a medical device that would make patient compliance easy.
“People with diabetes are prone to numerous complications: patients have to monitor their blood glucose level constantly to avoid hyperglycaemia: the procedure is usually done by pricking the finger and drawing blood for analysis. You can only imagine how uncomfortable it is for young children or their parents who sometimes have to wake their child up several times a night to check it. Being involved in a company that develops a device capable of making so many people’s lives easy is a subject of great pride for me”, she says.
Dalia Argaman is currently in charge of clinical regulatory affairs and quality assurance in Earlysense. The company develops contactless sensors that are placed under hospital bed mattresses and allow to monitor vital signs (heart rate, respiratory rate and other parameters) in a contactless way without making the patient feel uncomfortable.
“They can help the physicians, nurses to continuously supervise the patient and detect early signs of deteriorating in order to intervene early, thus reaching a better outcome”. All of this is done using passive monitoring of the patient’s movement and without requiring patient compliance.
“There is currently a long delay between the time that a medical device is being developed by research and development teams, execution of medical device clinical trials, analysis of data received from clinical data management team, submission to FDA and the time that products get to the end users. The delay is connected with vigorous testing a product has to get through in order to be in compliance with standards and be approved. I think FDA understands well the importance of using automation to accelerate the process of executing clinical trials in order enable these medical devices to get to market and start saving lives sooner”.
Invisible gorillas and detection of adverse events in medical device trials
What is easier to detect in your study – Slow-moving or fast moving deviations?
This post considers human frailty and strengths.
We recently performed a retrospective study of the efficacy of Flaskdata.io automated study monitoring in orthopedic trials. An important consideration was the ability to monitor patients who had received an implant and were on a long term follow-up program. Conceptually, monitoring small numbers of slow-moving, high-risk events is almost impossible to do manually since we miss a lot of what goes on around us, and we have no idea that we are missing so much. See the invisible gorilla experiment for an example.
One of patients in the study had received a spinal implant and was on a 6 month follow-up program dived into a pool to swim a few laps and died by drowning despite being a strong swimmer. Apparently, the pain caused by movement of the insert resulted in loss of control and a severe adverse event. The patient had disregarded instructions regarding strenuous physical activity and the results were disastrous.
It seems to me that better communications with the patients in the medical device study could have improved their level of awareness of safety and risk and perhaps avoided an unnecessary and tragic event.
Subjects and study monitors are both people.
This might be a trivial observation but I am going to say it anyhow, because there are lessons to be learned by framing patients and monitors as people instead of investigation subjects and process managers.
People are the specialists in their personal experience, the clinical operations team are the specialists in the clinical trial protocol. Let’s not forget that subjects and study monitors are both people.
Relating to patients in a blinded study as subjects without feelings or experience is problematic. We can relate to patients in a personal way without breaking the double blinding and improve their therapeutic experience and their safety.
We should relate to study monitors in a personal way as well, by providing them with great tools for remote monitoring and enable them to prioritize their time on important areas such as dosing violations and sites that need more training. We can use analytics of online data from the EDC, ePRO and eSource and connected medical devices in order to enhance and better utilize clinical operations teams’ expertise in process and procedure.
A ‘patient-centered’ approach to medical device clinical trials
In conditions such as Parkinsons Disease, support group meetings and online sharing are used to stay on top of medication, side effects, falls and general feeling of the patient even though the decisions on the treatment plan need to be made by an expert neurologist / principal investigator and oversight of protocol violations and adverse events is performed by the clinical operations team. There are many medical conditions where patients can benefit by taking a more involved role in the study. One common example is carpal tunnel syndrome.
According to the findings of an August 3rd, 2011 issue of the Journal of Bone and Joint Surgery (JBJS), patients receiving treatment for carpal tunnel syndrome (CTS) prefer to play a more collaborative role when it comes to making decisions about their medical or surgical care.
Treatment of carpal-tunnel syndrome which is very common and also extremely dependent upon patient behavior and compliance is a great example of the effectiveness of “shared decision-making, or collaborative, model” in medicine, in which the physician and patient make the decision together and exchange medical and other information related to the patient’s health.
As the article in JBJS concludes:
“This study shows the majority of patients wanted to share decision-making with their physicians, and patients should feel comfortable asking questions and expressing their preferences regarding care. Patient-centered care emphasizes the incorporation of individual styles of decision making to provide a more patient-centered consultation,” Dr. Gong added.
In a ‘patient-centered’ approach to medical device clinical trials, patients’ cultural traditions, personal preferences and values, family situations, social circumstances and lifestyles are considered in the decision-making process.
Automated patient compliance monitoring with tools such as Flaskdata.io are a great way to create a feedback loop of medical device clinical data collection, risk signatures improvement, detection of critical signals and communications of information to patients. Conversely, automated real-time patient compliance monitoring is a a great way of enhancing clinical operations team expertise.
Patients and study monitors are both people.
Why paper is not an option for your medical device clinical trial
This is a piece David wrote a couple of years ago originally entitled “Why you cannot afford to use paper in your first Phase I efficacy trial for your medical device”. David’s premise is that people do not like change.
In all walks of life, people do not like change.
We have heard the axiom change is good all throughout our lives, but the fact remains that people, as basic animals, are hesitant to embrace change and take on new endeavors. Human beings are creatures of habit, and are more often than not content within their comfort zones, regardless if they are losing out on valuable experiences, money, etc.
Studies have even been conducted revealing how opposed to change we creatures of habit are. People will sacrifice the opportunity to enhance their quality of life because it may require a change to their routine, or learning new habits, and humans hate that.
In the clinical research industry, paper-based data capture methods have been used effectively, and for decades. Paper is the norm, and many a successful study have been conducted using this such method. While paper is the traditional, tried-and-true method for data capture and management (especially during Phase I efficacy trials of medical devices, which typically have smaller subject counts and shorter study durations), that does not mean it is the best method available, or that it is the most cost efficient.
In fact, the last point is no longer true whatsoever.
There are small CROs and clinical study sponsors that are so used to paper data capture for small medical device clinical trials, that they oppose the change to electronic data capture (EDC). However, while some early objections were valid in opposition of EDC for Phase I, they no longer ring true.
EDC has been implemented for clinical studies, particularly in later-stage trials such as Phase III studies with thousands of patients, for a little over 15 years. By now, many of the concerns regarding the ample paper vs EDC debate at any clinical study stage are now moot.
Today we are going to touch upon why one cannot afford to use paper for Phase I efficacy trials for medical devices, and will greatly benefit from the change to cloud EDC.
Time savings on amendments
During Phase I efficacy testing, pharmaceutical companies are getting their feet wet for the first time while developing a new drug. This is the stage with the highest level of patient risk, and EDC quickly thwarts paper-based systems in this realm.
Phase I experiences the most amendments to drug administration frequency, dosing, and amendments to trial need to always be compliant with the FDA’s 21 CFR Part 11. Vendor validated EDC systems are easily augmented to comply with changes to FDA regulations, and have measures in place within the software, to monitor and ensure that study SOPs are compliant every step of the way.
Paper simply cannot do that.
Also, amendments, whether at the hands of a regulatory agency or medical device company, tack on months of extended study time and costs. According to a study done by Tufts University, a single amendment using paper-based systems increases study time by an average of 2 months and costs the study over $400,000. The study also showed that on average, each study experiences 2.3 amendments to protocol.
The time savings, and thus cost savings, on amending SOPs is enormous for studies conducted using EDC, as the software and eCRFs can be augmented in the blink of an eye. Also, if further amendments are required down the study chain, they are made just as quickly.
Real-time data monitoring of cleaner, faster study data
Using EDC instead of paper affords clinicians and data monitors real-time access to data capture. Also, cloud EDC like Clear Clinica is mobile accessible, so all members of the study team can remotely access data using mobile devices like smartphones or tablets, the very second it is entered. This is especially valuable for Phase I trials, because these have the highest risk to study subjects, as they are the first in line to test the drugs.
Even though Phase I efficacy trials do not typically involve hundreds of subjects, adjustment to treatment protocol and dosing need to be made with as little delay possible. Patient safety is a top priority for clinical studies.
For studies using patient reported outcomes (ePRO), EDC wins over paper-based systems every time. When a patient enters data into the system, risk-based monitoring protocols within the software inform study teams whether or not a patient is at risk for harming their health if they proceed at the administered dose. This allows clinicians to make adjustments to dosage, or cease the subjects participation in the study, before their health is harmed if the dosage is too high or if grossly adverse effects are experienced by the patient.
Again, paper simply cannot perform in this manner for Phase I efficacy testing.
Further, using EDC for Phase I is smarter than paper regarding cleaner, error-free data. Human error occurs. Even the brightest of clinicians and data monitors will make a mistake when entering data, or miss an incomplete form. Especially being that Phase I is the first stage for drug testing, the cleaner data is from the get-go the smoother it will be for conducting Phase II and III of the trial.
The EDC system software can be set so that eCRF values are standardized, so that when data is captured and entered into the eCRF fields error notifications are displayed when data is outside of the field parameters. The same goes for submission of eCRFs that are incomplete. EDC systems like Clear Clinica are programmed according to data parameters set by sponsor or CRO staff for each trial’s needs. Also, once the eCRF parameters are set, they can be modified if needed according to amendments, but otherwise they remain uniform, saving time during the entire study cycle’s lifetime.
You cannot afford to not use EDC
Nobody will deny that the up-front costs of implementing cloud EDC for Phase I will cost more than a paper-based system. However, not doing so because of that reason is myopic and short-sighted. The safety, risk, time, and data quality savings are well worth the initial investment, as the system is not going to be used for only one study.
Down the road, after incorporating an EDC for Phase I, and using it for II and III, the money spent is quickly offset by the costs saved on time, IT personnel expenses (EDC vendors have support staff to solve whatever issues may arise), and data assurance, amongst others. The sooner you switch to EDC, the sooner future studies will save your CRO or sponsor organization money, and mitigate Phase I patient risks.
A structured 7 step process for risk assessment of a medical device clinical trial
In this essay, I discuss a systematic methodology for evaluating risk in your medical device clinical trial. This is a methodology that has proven itself in hundreds of security and privacy compliance risk assessment projects in a wide variety of healthcare, clinical and IT scenarios.
It is a given that the people charged with your clinical trial planning,regulatory affairs and operations are better at executing standard operating procedures then in performing risk analysis and thinking like attackers.
Risk assessment is a process that starts before you write the protocol, when you are writing the CRF (to determine what data to collect) and any time there are amendments to the study.
See the below graphic from the Transcelerate Web site to see why procedures do not protect your clinical trial and why SDV does not assure patient compliance to the protocol. Note that “Material risk” is any threat to the success of the study from problems with study startup to problems with poor patient compliance.
Does counting compliance activities secure the deliverables of your clinical trial?
First define “secure”.
Security is about reducing the impact of unpredictable attacks on assets – in your case, attacks on the 2 most critical assets of your clinical trial – the data and the subjects.
Some examples of unpredictable attacks on your clinical trial:
There may be multiple sources of data errors at sites, ranging from mistakes, misunderstandings, sloppiness and all the way to incompetence.
There may data fraud – deliberate fabrication or falsification of data
There are patients that comply and patients that take their treatment randomly and in strange and wonderful ways.
There are patient reported outcomes that make sense and then there are the people who write War and Peace in the ePRO system and crash the SAS analysis program with special characters they used.
Will compliance activity check-boxing mitigate ANY of the above attacks?
How to mitigate unexpected attacks on your data and patients
Once we understand that check-box compliance procedures are not a good countermeasure for threats to your study deliverables (solid scientific data, patient safety, patient compliance with the clinical protocol) what are our options for mitigation?
Consider your strengths and weaknesses.
Starting with your weaknesses, it is a given that the people charged with your clinical trial planning,regulatory affairs and operations are better at executing standard operating procedures then in performing risk analysis and thinking like attackers.
There is a fundamental divide, a metaphorical valley of death of mentality and skill sets between a regulatory-affairs and clinical operations mindset and a professional security mindset.
This essay offers a systematic approach – if you will, a common language, a language of people-centric threat modeling that helps clinical managers cross the chasm between thinking like a regulatory affairs person and thinking like an attacker who wants to destroy your study.
Start by thinking about how your study can be attacked.
Analyzing the impact of attacks on medtech studies requires hard work, hard data collection and hard analysis. It’s not a sexy, fun to use, feel-good application like Apple Music. Risk analysis may yield results that are not career enhancing, and as the threats get deeper and wider with bigger and more complex trials – so the security valley of death deepens and gets more untraversable.
There is a joke about systems programmers – they have heard that there are real users out there, actually running applications on their systems – but they know it’s only an urban legend. Like any joke, it has a grain of truth. IT and security are primarily systems and procedures-oriented instead of customer-safety oriented. Similarly – clinical regulatory affairs are primarily paper and process-oriented instead of attack-oriented.
Leave your paper and process comfort zone
If the essence of security is protecting the people who use a company’s products and services then the essence of security for a clinical trial is protecting patients and acquiring reliable data.
A structured 7-step process for risk analysis of your clinical trial
We propose a structured process for risk analysis and ongoing risk management. No previous training is required and the process can become a key part of a medtech developer’s management toolkit.
The risk analysis and management process has 7 steps as described in the below schematic (“the risk management loop”). The process uses threat modeling and quantitative risk assessment methods based on providing a financial value to assets (such as EDC systems and patient eCRF records) in order to determine value at risk and prioritize security countermeasures.
The 7 step risk process provides a systematic way to manage risk while responding to changes in regulation, business environment and clinical research feature set/functionality. Let’s start with some basic definitions:
Vulnerability is a weakness, limitation or a defect in one or more of the system’s elements that can be exploited to disrupt the normal functionality of the system. The weakness or defect may be either in specific areas of the system, its layout, its users, operators, and/or in its policies and procedures.
Countermeasure is a technical, physical or procedural safeguard that mitigates one or more vulnerabilities.
Asset – data, systems, physical assets or intellectual property of value to the organization.
Threat – action(s) that exploit vulnerabilities in order to damage assets.
Asset value – the financial value of an asset that is destroyed of stolen. Assets may be digital (software source, physical (a server) or commercial (a corporate brand).
Damage to Asset – damage to a physical asset or damage to a digital asset in terms of breach of confidentiality, impacted system availability or broken integrity of systems and/or data. Damage is estimated in financial terms.
Threat probability is the likelihood that a threat will turn into a real attack. Threat probability can be described in terms of ARO – Annual Rate of Occurrence; i.e. how many times a year that the attack is forecasted to happen.
Threat risk is the likelihood of damage that may be caused to one or more assets by the threat. Recommended countermeasures the possible countermeasures that reduce the threat’s risk based on the countermeasures that mitigate the threat vulnerabilities.
Actual countermeasures (aka mitigation plan) is a subset of recommended countermeasures that is assumed to be the most effective for mitigating a specific threat. Choice of specific safeguards is often a judgment call of the threat analyst.
Countermeasure cost is the financial value that is associated with the implementation of a specific countermeasure.
Countermeasure cost effectiveness is the degree of mitigation introduced by a specific countermeasure to the overall risk in the system in relation with the cost of implementing this specific countermeasure.
Attacker is a person (or group of persons) that may perform the steps of a specific threat scenario.
Attacker Types are the various classes of attackers that are differentiated according to their motivation, qualification, available attack tools and their accessibility to the attacked system’s resources.
Entry Points are points of entry made by attackers into the system, for example doors in a building or users who have a login to your EDC system.
The 7 step risk analysis loop
Risk analysis is not a one-way, one-time process you do, report and file away. Analyzing attacks and risk in your studies is an ongoing exercise always relying on quality human intel from the field – from CRCs, subjects and site monitors.
Step 1 Set scope
The threat analyst(s) will identify reasonable threat scenarios and their probability.
Read this if you are new to risk analysis
Choose one (1) question you want to answer. That’s it. Only one (1). For example – “what is the threat scenario for patients participating in the study and not passing inclusion/exclusion criteria”? After you have nailed the question, nail the threat scenario – i.e. how it can happen. After you nail the threat scenario – quantify the threat in terms of probability of occurrence and its impact and potential damage to your study.
Read this if you are a medtech developer
In a medtech study which uses wearables, connected medical devices or mobile medical device apps (or any combination thereof), having up-to-date documentation of software functionality and architecture is required in order to correctly identify vulnerabilities and threat scenarios. The following documentation is required as part of the risk analysis process:
1. Functional description of the system including relevant use cases
2. Architectural diagram of the system
3. Documentation of sub-modules
How to document the risk assessment for your medical device study
Up-to-date documentation of the study protocol and CRF is required in order to correctly identify vulnerabilities and threat scenarios. Historical records of protocol amendments is unnecessary.
The following source documentation is required as part of the risk analysis process:
Treatment schedule and visit flow
CRF edit checks
These documents must be detailed enough to be used as reference for the decisions regarding the applicability of various threat scenarios to the analyzed system.
Step 2 Identify assets of your study
The correct mapping of assets (EDC database, patient safety, drug accountability data, etc), their financial value and the evaluation of financial loss to the sponsor when these assets are damaged or stolen, is one of the most critical tasks in the threat analysis process. The assets value is used as the basis for calculating threat risks and countermeasures priorities.
Asset valuation is not a one-time activity
Due to the importance of asset valuation, the asset list and corresponding values should be reviewed once a year by the controller or CEO during the course of the study.
Step 3 Identify the moving parts (components) in your study
Using a systems approach to your study, map the moving parts in your study. This will include application software components (EDC, IWRS, ePRO, centralized monitoring systems etc), people functions (study monitors, site monitors, project manager, CRCs, principal investigators).
Map the “moving part” entities to assets (for example patient records) and update the threat model with the components and functions.
Tagging different components and functions in the system help the analyst in classifying the various data and software entities and relating them to specific vulnerabilities and safeguards such as protecting PHI processed by an outsourced call center.
Step 4 Identify your study vulnerabilities
Identifying and classifying vulnerabilities requires the analyst to be intimate with the study primary and secondary endpoints, safety endpoints, protocol design, implementation and deployment details. The analyst should also be familiar with clinical operations procedures and the types of users, customers and patients that use the system or are involved with delivering services.
Step 5 Build / update the threat model
Classifying attacker types
The basic attacker types are: study user roles (site and study monitors,Pis, CRC, project managers, IT staff or cloud EDC providers) , malicious outsiders, trusted insiders and other site staff and outsourcing service providers. Additional attacker types (such as hacktivists) may be added when relevant. Different attacker types will have different motivations and different costs for mounting an attack. Attack motivation and cost are an important part in estimating threat probability since cheap attacks by highly motivated individuals are more likely than expensive attacks by attackers with little to gain.
Identifying attack entry points
The best strategy for this step is to review attacker types and document every possible way a potential attackers could access the system. The list of entry points may be refined in the course of the risk management loop.
Step 6 Build your risk mitigation plan. Calculate residual risk
Risk assessment is not over until the fat lady sings. You walk away from the risk assessment table with a much deeper understanding of what threats count and how much residual risk you have after deploying controls – technical controls, monitoring of deliverables, patient safety monitoring
This is the most important step of the risk analysis and management process. The outputs are:
A map of the relationships between threats and area tags, assets, attacker types, entry points and vulnerabilities
An evaluation of the total damage and risk parameters for each of the threats
Write mitigation plans
Calculate residual risk – i.e. how much risk exists after you implement your new controls.
Since threats are the most complex entities in the model, the process of identifying and constructing threat’s elements and parameters has a ‘decomposition’ flavor. During this step the analyst(s) will have to return to previous analysis steps in order to create missing entities, such as assets and vulnerabilities that are referenced by the threat that is constructed.
Step 7 Validate your findings
The accurate identification of countermeasures and their relations with vulnerabilities is the basis for validating the correctness of the risk mitigation plan. The best way in our experience of validating a risk analysis is to show it other people outside your office and ask them what they think.
A list of countermeasures that mitigate vulnerabilities: The list should include the implementation cost and an indication if the countermeasure is already implemented.
A map of the relationships between countermeasures and vulnerabilities: This map shows which vulnerability is mitigated by which countermeasure(s).
A validated risk mitigation plan will include the following management level reports:
Threats ordered by risk
Threats ordered by the financial damage
Safeguards ordered by risk mitigation percentages
Safeguards ordered by their effectiveness (mitigation/implementation cost)
Asset value at risk before mitigation
Residual value at risk after the mitigation plan
We have presented a systematic 7 step process for identifying and analyzing threats to the assets of your clinical trial – whether its unpredictable user behavior or patients at risk.
Assessing the risk posture of any study will benefit from this proven systematic methodology and will help you take a paper and process-oriented study team from a place of weekly and monthly reports and activity-counting to a faster-moving, and vastly more effective place of risk understanding and mitigation.
100X faster to deviation detection in medical device studies.
Patient compliance automation on the flaskdata.io platform for a medical device clinical trial is 100X faster than manual monitoring.
Automated compliance monitoring analytics and real-time alerts let you focus your site monitoring visits on work with the PI and site coordinators to take total ownership and have the right training and tools to meet their patient recruitment and patient compliance goals.