Tag: CFAST

Dear CFAST:  Please Slow Down to Catch Up

Wayne KubickLeave a comment

A new year and many new things to think about in the greater world of research and healthcare, though there are still plenty of unfinished thoughts about CDISC to expound upon.  In my last post, I tried to sum up my thoughts on SDTM, advocating a more restrained approach that prioritized stability of the core SDTM/SDTMIG standard over frequent change, and proposing a cookbook recipe approach to addressing gaps in the model, especially for the CFAST Therapeutic Area User Guides (TAUGs).

And today, I’d like to echo the slow food movement, with a plea directly to CFAST to also take a breather and slow down. This may seem like a departure from my past point of view – I was a strong advocate of adopting Scrum for standards in order to get more things done. But moving ahead more rapidly should not be done at the expense of consistency and clarity of approach.   CFAST TAUGs have been produced at a relatively prolific rate of 6 or more per year since 2013.  Yet the pressure during that span has been to issue even more TAUGs each year, rather than maintaining consistency in the modeling approaches used by them all.

So, as former Chair of the CFAST TA Steering Committee, I’d like to respectfully suggest that they apply the brakes for a bit until they can go back and bring the full body of published TAUGs to a common parity by fixing the inconsistencies that currently exist.

This is the long-standing “maintenance issue” that was never completely addressed during my term as Chair.  There are several cases where modeling approaches adopted in one TAUG changed when a new TAUG was issued, which might not be apparent to those who don’t read every separate publication.  These inconsistencies in approach work against the goals of standardization, and can only aggravate the types of variances between submissions that have often frustrated FDA reviewers with SDTM submissions to date.  Now’s the time to go back and fix these dangling loose ends, since the FDA has yet to issue a clear message endorsing TAUGs, and since many sponsors are still evaluating how to utilize them.

To take one prominent example, there have been at least four different approaches to represent the diagnosis date of the condition or disease under study in TAUGs over the years:

  1. Putting the date of diagnosis in q Medical History (MH) domain record, using MHCAT = ‘PRIMARY DIAGNOSIS’ and MHSCAT to distinguish between an onset course and a current course (though these important categorization values are not specified as controlled terminology). This approach was used in the Alzheimer’s, Parkinson’s and other TAUGs.
  2. Creating a Findings About (FA) record, with the FAOBJ = <disease or condition>, and the date as an observation result. This approach was used in the Asthma TAUG.
  3. Creating a supplemental qualifier record with QNAM=’MHDXDTC’, an approach that was used in the Diabetes TAUG.
  4. Creating a new MH record using the new SDTM v1.5 variable MHEVTTYP = ’DIAGNOSIS’ – which was an approach recommended by the SDS Leadership group (including me) in 2014 and was adopted as the preferred approach in some of the more recent TAUGs (e.g., MS). Unfortunately, adoption of this approach has not been embraced by the full SDS team and is limited because it’s not included in the SDTMIG and requires a new variable which is not in the currently accepted v1.4 SDTM.

Now, given that the date of diagnosis is likely to be of interest to all medical reviewers, the notion of how to represent a date of diagnosis of the disease under study consistently should be tackled independently of any TA.  And the correct approach should be easy to find by implementers.  But, currently, the method may vary depending on what document each implementer uses as a guideline.

One way for dealing with this type of maintenance issue is to remove it from the individual TAUGs and making it a standardized convention that could be consulted by any implementer as a separate resource.  Using my cookbook recipe approach, such a guideline might say:

  • Always represent Diagnosis Date as an MH event record
  • Use MHEVTTYP to distinguish it from other elements of history with controlled terminology value = ‘DIAGNOSIS’.
  • If using SDTM v1.4 or earlier, represent MHEVTTYP (with controlled terminology QNAM value of ‘MHEVTTYP’ and QVAL = ‘DIAGNOSIS’.

Now (especially if supplemental qualifiers are represented within the parent domain where they belong) wouldn’t that be easier on the FDA reviewer?  Wouldn’t it be helpful to always know where to find such an important bit of data no matter who submits it irrespective of which reference (IG, UG) an individual sponsor employee may have chosen to follow at the time?  I mean, can CDISC and CFAST really say they has an effective standard if they’re not consistent on such points?

So, one more shout in the dark:

  • Let’s ease the constant pressure to create new TAUGs, domain models and new IG versions (which often seem like they’ll never be finished anyway)
  • Stop creating new TAUGs that contradict modeling approaches of older ones until all the published ones can be made consistent to a common baseline. One way to do this would be to remove all references how to represent certain modeling use cases like Diagnosis Date from the published TAUGs and replace with a link to a document showing a preferred standard approach (though an example might still be provided in the TAUG).
  • Start representing more recipes and examples for other preferred modeling conventions with better engagement of the CDISC standards implementer community as an interactive forum.
  • Make a firm decision once and for all between “Taste Great” and “Less Filling” as key modeling principles moving forward.

If you’d like to learn more about some of the significant variations across the CFAST TAUGs, please refer to this excellent paper by Johannes Ulander and Niels Both presented at PhUSE 2015.

Advertisement

A Call to CDISC Arms – Recapping My SDTM Posts

Wayne Kubick4 Comments

I’ve been posting a series of thoughts on the CDISC SDTM over the past few weeks, and realize that most people won’t have time to read them all, so thought I’d sum up with these CliffsNotes before moving on to other thoughts for improving clinical research.  So, as former CDISC CTO, here are my essential hindsight thoughts in a nutshell on what should be done with the SDTMIG:

  1. Prioritize stability for the SDTMIG so the world can catch up. In SDTM as a Cookbook I make a plea to aim for more SDTMIG stability by minimizing new versions, suggesting that new domains be presented as recipes to be crafted from the SDTM rather than as fixed domain models in the IG.  I propose a leaner, more flexible approach to applying the SDTM to create new custom domains for data not previously addressed in the current SDTMIG, a suggestion which should be adopted by TAUGs immediately.
  2. Fix supplemental qualifiers now! RIP Time for Supplemental Qualifiers makes another plea to represent non-standard variables (NSVs) in the parent domain where they belong, using Define-xml tags.  Acknowledging past FDA concerns about file size, suggests an alternative of representing these as a separate file that can be easily merged onto the parent as extra columns (much more easily than the vertical method in SUPPXX files today).  This has been a nuisance for far too long – fix it.
  3. Hit the brakes on creating so many new domains. As in the “Cookbook” blog,  The Lumping and Splitting Acid Test makes another plea to be more conservative in creation of new SDTMIG domains, proposing a default “lumping” approach which should only support new domains when there’s no ambiguity about what data to put there.  The more domains we give adopters to choose from, the more likely that they’ll make different decisions – the exact opposite of what a standard should do.  It’s also time to realize that such granular separation of domains probably won’t be the way of the future with patient data in integrated data repositories – see the FDA’s Janus CTR for an example.
  4. Describe use cases for applying the SDTM instead of IG version updates. A Recipe from the SDTM Cookbook gives an example of creating a new domain following the SDTM as a Cookbook approach.   This can be done without creating new IG versions, meaning that people can actually get accurate new information more quickly instead of waiting years for the next IG version.
  5. Time to plan the next generation. Life after the v3.x SDTMIG offers yet another plea to refrain from creating too many new versions of the SDTMIG v3.x series, and begin work instead on a next generation v2 SDTM and, subsequently, v4 SDTMIG that fixes long-standing problems and limitations and is more compatible with interoperability goals and newer technologies.
  6. Clarify the role of CFAST Therapeutic Area User Guides. No blog on this yet, but it’s essential to clarify that CFAST Therapeutic Area User Guides should NOT be considered as separate new standards – they should be viewed as ways to apply current standards. We can provide new examples of how to use a standard whenever a valid new example is created – but we simply can’t allow more confusion to be introduced into an industry still learning how to use our standards by frequently pushing out conflicting information or confusing directions about draft domains or modeling approaches that don’t conform to current standards acceptable to regulators.  My feeling is that TAUGs should focus on describing use cases and recipes to apply the current standards as well as identify new terminology to be applied.  Of course, would be better if we fixed a few things first – notably supplemental qualifiers and probably the disease milestones approach introduced with the Diabetes TAUG.  So maybe a v3.3 SDTMIG is necessary – but let’s draw the 3.x line there.

As we get clearer to the onset date of the binding guidance, we need to think differently if we want to make implementation of data standards the success story it ought to be.  So (maybe after finishing v3.3 soon, I hope) let’s take a breather, let the soup simmer awhile.  Meanwhile, let’s arouse the community to share their implementation challenges and experiences more openly and widely (as some of the User Groups already do, with a special mention to the energetic example set by the CDISC Japan User Group).  Let’s agree we need to gather more common practical experience and knowledge before we jump to develop new disruptive content.

And let’s get going collecting those requirements for the next, great leap forward instead.

That’s the pitch, so let the cards and rotten tomatoes fly.

A Recipe from the SDTM Cookbook

Wayne Kubick5 Comments

In my earlier posting on SDTM as a Cookbook, I described an alternative approach for defining new domain models for use with CFAST Therapeutic Area User Guides (TAUGs). Based on an internal poll of SDS team members, there seems to be a desire to create many domain models (a predilection toward splitting, rather than the lumping approach I favor).  Yet creating new domains is a frustrating and lengthy process. Although these are now mostly being modeled by CFAST teams with very specific use cases, there has been a tendency to also vet them through SDS in a more generalized form as part of a batch associated with a future SDTMIG release, a process which can take 2-3 years or more.  In the meantime, TAUGs are faced with proposing draft domain models under a stricter timeline, well before they exist in the officially sanctioned normative SDTMIG world.

What a waste — and it gets worse.  Once the domain is issued as final as part of an SDTMIG version update (indeed, that’s assuming the SDS team consensus allows it to and it actually passes the comment process) it now has to be evaluated by FDA before they can determine whether they’re ready to accept it.   Although 15 TAUGs have been posted to date, the FDA has yet to clearly indicate their readiness to accept any of them.  And the acceptance process has also been excruciatingly long (it took nearly 2 years for FDA to announce readiness to accept SDTMIG v3.2 – even then with some restrictions). In the meantime, people simply make up some other approach to get their daily work done – the antithesis of standards.

Let’s take a current example of how we may have applied the cookbook approach to draft domains included with the just-posted TAUG-TBv2. This TAUG includes 5 new draft domains as well as revisions to 3 existing domains which are presented as SDTMIG domain models.  One of the new domains (which was vetted with SDTMIG v3.3 Batch 2 and also used in Asthma but not yet released as final) is the RE (Respiratory Physiology) domain. This is a Findings General Class domain, which is mostly consistent with the current version of the SDTM v1.4 except for the addition of 3 new variables:  REORREF, RESTREFN and REIRESFL. (An earlier version of this domain was also included in the Asthma and Influenza TAUGs.)

Now a cookbook recipe might present this RE domain as a list of steps to follow to “roll your own” domain.  Instructions might include:

  1. Create a new custom domain with the standard identifiers and all variables from the SDTM v1.4 Findings General Class.
  2. Assign the Domain Code and prefix “RE” from controlled terminology.
  3. Insert the standard timing variables that are typically provided with a Findings Physiology domain.
  4. Create the following 3 new Non Standard Variables (NSVs):
    1. REORREF
    2. RESTREFN
    3. REIRESFL

(Note definitions for these might be pulled from a newer version of the SDTM which is now being updated more frequently, or else from a CDISC Wiki resource).

  1. Remove any unnecessary or irrelevant permissible variables such as REMODIFY, RETSTDTL, etc. – just like you do with published domains. (Note that these are all permissible variables – assigning a new NSV as Required or Expected would be a complication, but this would be an odd choice for a newly created variable anyway).
  2. Add any additional NSVs in the usual manner (this would be much smoother if the new proposed method of putting NSVs in the parent domain was adopted).
  3. Apply other controlled terminology bindings for variables within the domain (such as RECAT, RETESTCD, etc.  that are declared in a sample define.xml file that’s posted along with the recipe.

As the output of this exercise one would normally create the define.xml metadata for the domain as well as a RE.XPT file (which will later be populated with data values).   The sample define file included with the recipe would also specify which controlled terminologies apply to both the standard and new non-standard variables (I assume that new terminology values that would be intended for use in this domain would be created through the normal terminology request process, and simply referenced in the define-xml example). The recipe could still provide a draft domain in the usual Word Table or Excel format – but this would be presented as an example rather than a normative specification, similar to including an illustration or photo in a recipe.

I believe it should be sufficient to apply the standard class-level validation rules (which include checking for controlled terminology assignments), which can be addressed separately from the domain model, so there should not be any specific new user acceptance testing required by FDA. FDA might also specify separate content-based checks they may want, but these can be added at any time later, once they’ve had a chance to review submissions using this model.  But new rules can also be added outside the IG.  And while it will be technically a non-binding custom v3.2 domain in each submission, if it’s conforming to the recipe (which can be clearly stated in the define) it can serve the same purpose as a new SDTMIG domain in a future version. The difference is that it can be put directly into use.  A beneficial side effect is that this also encourages early testing among the research community, which might result in beneficial tweaks to the recipe, which can be maintained over time and augmented with more and more examples suggested by adopters in a crowd-sourced Wiki sharing environment, which should only serve to make the domain model more solid over time.   Sure, this might require review and curation by the SDS team, but that should be a lot less onerous than the current process.

The benefits of such an approach include:

  • Making it simpler and easier to create new domain models based on existing published versions, which might help shorten the development time for TAUGs
  • Allowing sponsors to adopt these new models more rapidly without waiting for new domains or FDA announcements
  • Making it possible for FDA to accept these models without a lengthy acceptance process
  • Providing an improved, rapidly evolving Wiki-based knowledge resource to help sponsors address representation of data that doesn’t fit in existing final domains in a more consistent manner.
  • Minimizing the number of new versions of the SDTMIG that have to be handled by industry and regulatory authorities.

Of course, adopting such an approach is not trivial.  It would require buy-in by FDA and industry, and would require new methods for sharing these recipe guidelines (probably via the Wiki) and a whole lot of communication and training.  But it seems to me it would be a much more practical way to move forward to extend the reach of the SDTM for new TAs in a leaner, quicker manner with fewer maintenance and version management headaches.

The Lumping and Splitting Acid Test

Wayne Kubick10 Comments

Call me “Lumpy.”  And I’m a lumpaholic.  I acknowledge this predilection, because I simply feel lumping makes things easier, and I’m generally more comfortable with big pictures than detailed nuances.

But I’m not a lumping zealot.  I have many close friends are splitters, and I can respect their point of view some of the time.   It’s not unlike the dynamic between friends who are either Cubs or Sox fans in Chicago, for instance – we can agree to disagree, even when they’re wrong.

The term “lumping and splitting” apparently was coined by Charles Darwin with respect to how to classify species, and is explained nicely here:

  • (For lumpers) “two things are in the same category unless there is some convincing reason to divide them.”
  • (For splitters) “Two things are in different categories unless there is some convincing reason to unite them”.

This clearly defines the opening position of each camp, and relies on the existence of a compelling and logical reason to go the other way, which sounds reasonable enough.

Now a partisan divide of lumpers and splitters also exists in the CDISC SDS/SDTM microcosm, where the term is applied to the decision on whether to create new domains.   Lumpers believe there should be a limited number of domains, to make it easier for sponsors to know where to put data.  Splitters want to create many more domains with finer distinctions between them.  The CDISC SEND team follows a very fine-grained splitting approach.  But that does not necessarily mean human trials should follow the same pattern, since a lumping approach has also been followed with questionnaires and lab results data.  In the latter cases, the SDTMIG describes a way to split these often massive lumped domains into separate files to conform to FDA file limits, but they’re still considered the same domain.

This difference of opinion has recently been illuminated as the team has wrestled with how to represent morphology and physiology data, a topic that’s critical to CFAST Therapeutic Area standards.  Some years ago, the SDS team made a decision to separate physiology data by body system, and reserved some 15 separate physiological domain codes even though no specific use cases had been defined for all of those yet, while lumping together morphological observations in a single domain.  This proved problematic, because it wasn’t always clear which type of test belonged where.  So the team decided (through an internal poll) to merge Morphological observations into the various physiology domains.  An alternative lumping proposal to combine all physiology data in a single data, and use a separate variable for Body system (as was already the case in Events and the PE domain) was proposed but did not gain sufficient traction.

Splitting may be fine as long as there’s that convincing reason and it’s clear where to split – like the “Tear Here” perforation on a package.  We can easily see that AEs differ from Vital Signs (though the latter may indicate a potential AE).  And I’m not suggesting that we lump together domains previously released (well not all, anyway). But what do you do with a complex disease such as cancer or diabetes, or a traumatic injury that affects multiple body systems?  what happens with, say, observations related to a bullet wound that penetrates skin, muscle and  organs when you have to split these over multiple domains?

In such cases, wouldn’t a physician – or FDA medical reviewer – want to see all of the observations relevant to the patient’s disease state together rather than jumping around from file to file and trying to link them up?   And how are patient profile visualizations affected when multiple, possibly related morphological and physiological observations are split across many separate files, since patient profiles tend to look in a finite number of domains for core data (typically DM, EX, LB, VS, AE and CM) – adding in dozens of others is likely to be challenging.  And maybe it would reduce the constant stress of having to accommodate more and more new domains with each new version if SDS didn’t feel a need to keep adding more and more domains each time.

This brings me back to a “first principle” – making it easy on the reviewer. If you know exactly what you’re looking for, and you’re confident everyone else knows to put the same type of data in the same place, then maybe it’s easy to point to a separate fine-grained domain.  But what if you’re looking to see possible associations and relationships that may not be explicit (which FDA reviewers have lamented previously).  I think for most of us who are familiar with spreadsheets and other query and graphics tools, it’s far easier to have all the data you want in one place and rely on filter and search tools within a structured dataset rather than search/query across a directory full for files.  And that’s one reason why FDA has been working so long on moving from file based data review to their Janus Clinical Data Repository (CDR) – so reviewers can find the data they want through a single interface.  In my experience, CDRs (including Janus) do not usually split most findings type data by domain.

Lumping solutions should be more consistent between sponsors and studies, since there’s a simpler decision on where to put data.  Just like when shopping at Costco, it’s easier to make a decision when there are fewer choices involved. And just as it’s quicker and easier to pick out some bathroom reading from a single shelf rather than have to search through an entire library, lumping should make it quicker and easier to access the data you want.

So, what exactly is the acid test?   Whichever approach is chosen for SDTMIG in the future (and it should be up to the community to weigh in when the next comment period begins), one overriding principle must be in place:  a new domain split should never be created unless there’s a clear rationale for splitting (and in the case of Physiology, I don’t really see that myself), and it’s absolutely, unambiguously clear when the new domain is to be used for what kind of data.  If there’s any ambiguity about whether a certain type of data could go here or there, then we should opt for a lumping solution instead, and allow users to rely on query and filter display tools to pick out what they want.  Meanwhile, we can rely on our statistical programmers put data logically together in analysis files just as we always have.

So, lumpers of the world, unite!  There are simply too many other problems to tackle other than the “What domain do I use this time?” game.  Like defining rich metadata models for sets of variables with valid terminology (i.e., concepts), or defining a next generation SDTM that isn’t based on SAS XPT.

In the meantime, another scoop of mashed potatoes for me, please – lumps and all.

SDTM as a Cookbook

Wayne Kubick9 Comments

The original CDISC Study Data Tabulation Model Implementation Guide (SDTMIG) was built around a core set of domains describing data commonly used in most clinical trials — much of which were necessary to understand basic product safety.  Over time, the SDTMIG has been expanded with new versions that incorporated more and more domain models.  SDTMIG v3.2 has nearly 50 domains, with several more to come with the next version.

This domain-based approach, while very familiar to those involved in study data management and analysis, has been a mixed blessing in many ways:

  • FDA acceptance testing of new SDTMIG versions has been taking a year or more post publication, and sometimes this belated support for new versions comes with exceptions — a serious disincentive to early adoption by sponsors.
  • While continually adding new domain models may be helpful to some, others may find these threatening – due partly to the excessive effort of change management in a regulated systems environment in general and especially if they’ve already developed alternative approaches to modeling similar data in custom domains which might require last minute database changes to support new SDTMIG versions for ongoing drug development programs.
  • The time it takes the SDS team to release a new SDTMIG version is at least two years or more, and its already massive but steadily increasing size and complexity makes updating more difficult and adoption and understanding by end users burdensome. And still publicly available validation tools have not been able to keep up.
  • Meanwhile, the long timeline has made it necessary for the timeboxed CFAST Therapeutic Area (TA) User Guides to issue Draft or Provisional domain models to capture TA-specific requirements, which FDA seems reluctant to accept, again discouraging early adoption and injecting more fear, uncertainty and doubt since such domains may still undergo change before being added to a new SDTMIG version.

So this spiral of steadying increasing complexity and version management seems to be a cause of widespread consternation.

Cooking Up an Alternative Solution

What if we considered the SDTMIG as something more like a cookbook of recipes rather than a blueprint?  A recipe describes the general ingredients, but the cook has discretion to substitute or add some additional ingredients and season to taste.   What if the SDS and CFAST teams — rather than continuously creating more and more new domain models (sometimes with seemingly overlapping content), and waiting to embed these within a new version of the normative SDTMIG every few years to address any changes or additions —  took a more flexible, forgiving approach.  So a new Therapeutic Area User Guide would continue to describe what existing final SDTMIG domains may be most appropriate for representing TA data such as relevant disease history, baseline conditions, and, when necessary, include directions for cooking up a new custom domain for, say, key efficacy data.  The recipe would state which SDTM general class to use, which variables from that class apply, what current or new controlled terminologies should be used, and other implementation details, such as any incremental business rules that might be used for validation checks.  in lieu of a traditional domain model, the recipe could include an example of the Define-xml metadata for the domain.  But it’s up to the cook to adjust for the number of guests at the table, prepare, add garnishes and serve.  Such a recipe could be referenced (just like an SDTMIG version and a domain version) in a define-xml file (a proposed new v2.1 of Define-XML has features to support this) as if it was a standard, but, in reality, it would be a best practice guideline.

Representing domain models as recipes (or guidelines if you prefer) would have the advantage of producing domains that would already be acceptable for FDA submission (since custom domains are being accepted by FDA), and yet still being checked for consistency with controlled terminology and validation rules.  And adopters of CFAST TA standards might start using these more quickly, allowing these to be field tested so they can evolve more rapidly without waiting years for a new SDTMIG version to bless them.  As people learn more and gain more confidence, they can post additional examples and comments to a Wiki that can inform future user so everyone builds on everyone else’s experience over time.

Under this approach, the SDTM becomes the real model/basis for the standard, and the domain models would be treated as representative use cases.  The recipe can also include incremental validation rules, but must remain faithful to any SDTM model-level rules (which would have to be promoted upward from the IG presumably). New models may need new variables in many cases, and these variables should be added to newer versions of the SDTM.  But, assuming the SDS team finally adopts a more efficient manner of representing Non-Standard Variables (as they have already proposed in two drafts for comment) it would be easy enough for custom recipe-driven domains conforming to a current version of the SDTM to add each necessary new variable as an NSV at first.  These NSVs could then to uplifted to become standard variables later in a new SDTM version, which would only require a simple change to an attribute in the define.xml once that new version becomes available. Either way, the new domain model can be used immediately by applying current available structures in the SDTM with some incremental new terminology (such as a new domain code describing the contents and a new controlled term for a variable name.)

This does not mean the SDTMIG goes away – it’s still needed to describe the context, general assumptions and rules, and show how to implement key SDTM concepts like the 3 general classes, trial design model, special purpose and relationship classes.  But the IG wouldn’t need to cover each new variable and domain anymore and could focus on explaining core SDTM concepts with assumptions and rules instead.  It could make for a leaner SDTMIG, which doesn’t have to change so often, and impart more flexibility and agility in the use of domains.

Such an approach could also be a good first step toward decoupling the SDTMIG from the highly cumbersome, restrictive use of SAS v5 Transport files, and make the model more adaptable for implementation in new technological frameworks, such as HL7 FHIR and the semantic web.

Of course this still leaves a number of challenges in terms of terminology management, but that’s a topic for another day.

In the meantime, what do you think about a simpler SDTMIG, that emphasized applying the general model with terminology to meet the needs for new CFAST domains?