Compliance and agile success – another common hurdle

From a post for the SoftwareCPR.com website:

In a prior blog, we discussed a common hurdle to achieving agile benefits: focusing on isolated software process changes and not considering all four organizational factors.

Another common hurdle to agile success is thinking only about making agile “compliant.” You will miss the full power of agile if you do not consider the intent behind compliance activities and how agile can help better achieve intent.

Let’s start with the intent behind medical device software “compliance.” What’s the point? At the top level, it is building devices that fulfill intended use and do no harm to any user or patient. From a simplified compliance point of view, that means two things:

1. Assessing and mitigating any risk of harm before the device is used.
2. Establishing ability to determine if and how a device contributed to an actual harm or potential harm situation after the device is used. Corrections can then be made to prevent recurrence.

How can agile help?

Agile helps anticipate and mitigate risk by recognizing some fundamentals of human nature:

• Estimation limitations
• Procrastination (student syndrome)
• Comprehension limitations

Estimation

Humans can often do a reasonable job of estimation if, for example, the estimate:

• Is within their area of expertise
• Does not have to be 100% accurate/complete
• Is only for the short-term (a few weeks at most)

Unfortunately, deterministic-type development approaches (e.g., “waterfall”) assume predictability. Processes and controls are built around the ability to forecast accurately and completely for the entire project at the start. Changes once the project begins are abnormal rather than normal. Change control systems must be in place and followed if changes are needed.

Software practitioners know that near-perfect forecasting ability is the exception rather than the rule. Agile approaches are built on the opposite assumptions: there is unpredictability, and changes are normal, not abnormal. Agile processes and controls are therefore more PDCA (Plan, Do, Check, Adjust) oriented. Plans are considered assumptions rather than facts. In the “doing,” the assumptions are converted to facts that can be checked. The iterative and incremental nature of most agile approaches can therefore help with:

• Risks. Risks should still be estimated and mitigations planned at the beginning. This is no different from waterfall-type approaches except agile approaches expect change. Iterative development allows updating risks each cycle based on what is learned. We can both improve forecasts (e.g. add new risks, modify existing risks) and check how well our planned mitigations worked.
• User and patient behavior. We make assumptions about user and patient behavior when establishing intended use and planned features. Those assumptions will remain assumptions until users/patients interact “for real” with the device. Agile approaches typically incorporate incremental deliveries (or “experiments”) to get actual user/patient feedback. We can then use that feedback to confirm or change our assumptions. This allows updating our risks and mitigations based on real data, not assumptions.

Procrastination/student syndrome

“Student syndrome” is where someone begins a task at the last possible moment. The last possible moment is when the work “can” still get done in time (i.e., usually a small fraction of success probability left). The “syndrome” usually happens because the work is of low interest, difficult, a low personal priority, or some combination.

Documentation is typically not a favorite activity of software developers. It usually gets the student syndrome treatment, with documentation often done hurriedly at project end. At this point, memories are hazy and the urge to “just get something done” is strong. This can lead to missing risk discoveries, risk analysis improvements, as-built documentation, and so on.

The iterative/incremental approach in the estimation section above helps here, too. Including risk-related activities in a “Definition of Done” for user stories or the increment/Sprint can reduce student syndrome impacts.

An important note: doing something at the “last possible moment” is not the same as doing something at the “last responsible moment”! The latter is, in fact, an excellent strategy for decisions in particular. Better decisions can be made later when more is known and there is still time to act. Agile invokes the last responsible moment approach rather than the last possible moment approach.

Comprehension

Complexity adds time, cost, defects, etc. to projects in non-linear fashion. As complexity increases, the negative effects increase faster. There are also limits to how much complexity humans can comprehend at one time, increasing negative effects.

Agile approaches help by emphasizing incrementally building toward an end goal. Each increment is small and less complex, reducing the negative effects of complexity. This also makes estimates more reliable and useful by reducing variance. Estimates also improve faster as more is learned.

Actions to consider:

1. Help everyone on the team understand the intent behind compliance activities. This is especially needed when team members come from non-safety critical domains. Challenge them to think of leaner, more effective ways to achieve the intent within regulatory boundaries.
2. When implementing agile approaches, consider intent (#1 above) and how agile can help. AAMI Technical Information Report (TIR) 45, Guidance on the use of AGILE practices in the development of medical device software, can be helpful, especially for those new to medical device software.
3. Include risk-related activities in iteration (or component like User Story) Definition of Done.
4. Work “smaller,” building incrementally.

Need more? Here are some of the ways we can help:

• Explore SoftwareCPR.com for additional content and resources.
• Schedule to attend one of our compliant agile training sessions.
• Review your current process for any improvements with our help.
• Have our experts review a release for compliance.
• Accelerate your agile transition with strategic and/or targeted assistance.