Identifying Prospective Memory Deficits in MS with a Single Task from the Memory for Intentions Test: An IRT and ROC Analysis

Background: While many persons with multiple sclerosis (PwMS) have prospective memory (PM) issues, which can have an impact on everyday functioning, it is not routinely assessed. A single task that is sensitive and specific to PM deficits could be added to a screening battery, such as the abbreviated Minimal Assessment of Cognitive Function in MS (aMACFIMS), and thus improve detection and monitoring of these issues.

Objectives: To identify which of the eight tasks on the Memory for Intentions Test (MIST) has the best classification accuracy, sensitivity, and specificity for detecting PM deficits in PwMS.

Methods: Participants (n = 112) were PwMS who were part of a self-management study. PM was assessed with the MIST, with participants classified as impaired if they performed at the 1st percentile or below on the overall measure, per the test manual. Each task’s difficulty and discriminability were evaluated with item response theory (IRT) analyses with the R package MIRT. Five tasks with adequate difficulty and discriminability were further examined using receiver operating characteristic (ROC) analyses to determine their classification accuracies, sensitivities, and specificities. The tasks with the best classification accuracies were then compared in terms of their sensitivities and specificities using the R package DTComPair.

Results: Two tasks from the MIST had a classification accuracy of 90% and above in the current sample of PwMS: Trial 3 (100% sensitivity, 81% specificity) and Trial 4 (86% sensitivity, 90% specificity). These two tasks had comparable sensitivity (p = .317), with a trend towards Trial 4 having higher specificity (p = .061).

Conclusions: Trial 4 of the MIST, a verbal task with an event-based cue that requires participants to complete a pre-specified action after a 15 minute delay, has the optimal balance of sensitivity and specificity for detecting PM deficits in MS. Given the timing of its administration in the full MIST, it could potentially be added to the aMACFIMS. While Trial 3, a verbal task with an action-based cue and a two minute delay, also had strong classification accuracy, its ability to detect PM deficits in PwMS was likely influenced by the preceding cognitive load. The next steps will be to evaluate Trial 4’s utility as part of the aMACFIMS in terms of screening and monitoring PwMS’ PM deficits.