Findings and Projects

Summary of Active Memory related research being conducted at the School of Psychology, University of Sydney

Years:    2014 – 2016

Funds:   Australian Research Council Discovery Project DP140101147

Investigators: Robert Wood (University of Melbourne), Damian Birney (University of Sydney), Klaude Thomas (ABC)

 

Overview of Projects: The projects and studies reported below are those that have been completed or are in progress and have specific researchers delegated to them. There are numerous other projects not reported here planned for the next 12-18 months that draw on the Active Memory (AM) database. There are currently 7 research papers actively in preparation (with more planned); 5 under peer-review for publication; 2 published and 4 conference/symposium presentations.

 

Research Team and Collaborators:

The following have been involved with the project in various ways, and for that we are very grateful.

Damian Birney (Lead); Robert Wood (Lead CI); Klaude Thomas (PI, ABC); Sally Cripps, Vanessa Loh (postdoctoral fellow), Sophie Elwood (Senior RA), Kit Double (RA, PhD student), Shannon Webb (RA, PhD student), Joel Bateman (RA, PhD student), Danielle Gessler (RA), Natali Dilevski (RA, Honours student), Sarah Walker (RA), Bella Chu (RA), David Kohn (RA); the many staff at the ABC, including Tony Breckner (ABC); Joshua Crowley (ABC), Caroline Kinny-Lewis (ABC),, and all of the Active Memory participants, without whom our work would not have been possible.

 

Summary of projects, study type, participants and clinical measures.

  

Project 1: Cognitive Training in the “Wild”

Status: Completed

Sample: 810 Active Memory users (77% female, 18 to 92 years; mean = 61.96 years)

Assessments: ~ 1.5 – 2h per person (at home)

AM Training: At home as decided by user

Team: Damian Birney, Sophie Elwood, Vanessa Loh, Kit Double (PhD student)

Summary: We examined the psychological characteristics of AM users (e.g., personality, memory and reasoning self-concepts, everyday cognitive failures, and reasoning ability) who participate in online cognitive training under their own volition and direction (in the “wild”). A selection of AM users were invited to complete a 2h battery of personality and cognitive assessments, as well as a survey of background demographics and beliefs and attitudes toward learning. A follow-up survey was conducted at 6 months. AM training for these users was monitored over a 12 month period.

 

 

 

Project 2: Conversation Comprehension Task Validation

Sample: 60 university students (65% female, 18 to 35 years; mean = 19.5yrs); 100 AM users

Status: Completed; Data analyses being conducted

Assessments: 1h per person = 60h (in USYD labs)

AM Training: No training

Team: Damian Birney, Robert Wood, Vanessa Loh, Danielle Gessler (RA), Natali Dilevski (RA)

Summary: This study investigated the development of a test of conversation comprehension. One of the persistent problems in the field of cognitive training is assessing transfer of training to everyday activities. A series of 6 short (2min) video-recorded conversations between two people was presented to participants. Participants were then required to recall relevant and obscure information from the conversation. This task is used as a criterion measure in subsequent studies of AM users (Projects 3, 4, and 5).

 

 

Project 3: Active Memory at Work

Status: Completed

Sample: Melbourne Health (N = 36) and OPTUS (N= 20) employees

Facilitated Workshops: ½ day facilitated workshops (at organisation)

AM Assessments: 2 h per participant = 112 h (at home via AM)

AM Training: 15-20 mins, 5 x per week, for 4 weeks (at work)

Team: Robert Wood, Caroline Kinny-Lewis (ABC), Damian Birney, Vanessa Loh

Summary: The Active Memory At Work program comprised of a combination of facilitated workshops exploring attitudes and mindset toward learning and online AM training. We examined how AM training transfers to cognitive outcomes (administered through AM) and work-related outcomes.

 

 

 

Project 4: Blocked vs. Distributed Cognitive Training

Status: Data collection completed; Analyses being conducted

Sample: 60 students (complete); 65 Sydney-based participants drawn from AM users (complete)

Assessments (USYD labs): 2 (pre/post) x 2h per participant = 240h (students); 400hrs (AM users)

AM Training: 15-20 mins, 5 x per week, for 4 weeks (at home)

Publication: In preparation

Team:    Damian Birney, Vanessa Loh, Kit Double (PhD), Shannon Web (PhD), Danielle Gessler (RA), Natali Dilevski (RA), Sarah Walker (RA), Bella Chu (RA)

Summary: We examined how AM training outcomes are affected by different training schedules. Specifically, we compared the results when participants trained extensively on one AM task (massed training) before moving on to the next, compared with training on multiple tasks (distributed training) within a given session. Participants were novice Active Memory users based in the Sydney greater area who trained for 1 month. Each participant completed a 2h pre- and post-training cognitive and personality clinical assessment. The one-on-one assessments were conducted by our team in the labs at the USYD School of Psychology. Preliminary results suggest that training extensively on one task at a time results in more effective learning trajectories within Active Memory games, but there appear to be no differences between the groups in improvements on the cognitive battery.

 

 

 

Project 5: Behavioural and Physiological Alignment of Cognitive Training Outcomes

Status: Data Collection Completed; analyses of imaging data currently being conducted

Sample: 27 MRI-eligible community participants

Assessments (USYD psychology labs): 2 (pre/post) x 3h per participant = 162h

MRI Scanning time (USYD): 1h at pre- and post-training per participant = 54h

AM Training: 15-20 mins, 5 x per week, for 4 weeks (at home)

Team:    Psychology – Damian Birney, Vanessa Loh, Shannon Web (PhD), Danielle Gessler (RA), Natali Dilevski (RA), Sarah Walker (RA), Bella Chu (RA)

Brain Mind Center – Sharon Naismith, Loren Mowszowski, Shantel Duffy

Summary: We are using MRI technology to analyse the extent to which changes in brain volume and activation correlate with training performance and transfer gains. In addition, we are interested in how individual differences in personality as well as pre-existing neurological differences moderate the effectiveness of training and the extent to which neuroplasticity occurs. Participants were novice Active Memory users based in the Sydney greater area who trained for 1 month under massed vs. distributed schedules. Each participant completed a 3h pre- and post-training cognitive and personality clinical assessment and MRI scanning eligibility screening assessment. The one-on-one assessments were conducted by our team in the labs at the USYD School of Psychology. The pre- and post-training MRI scans were conducted at the USYD Brain Mind Centre.

 

 

Project 6: Longitudinal Investigations of Cognitive Training in Healthy Adults

Status: Ongoing

Required Sample: ~500 participants (current ~200)

Assessments: Initial 2h per participant = 1000h (at home); 3 x 1h follow-up per participant = 2000h (at home)

AM Training: at home as decided by user

Team:    Damian Birney, Kit Double (PhD)

Summary: We are performing a longitudinal study, which will follow Active Memory players over a 12-month period. Participants complete their regular Active Memory training and, in addition, every 3-months will complete a cognitive and self-report assessment on their cognitive function. This study aims to look at the long-term effects of cognitive training, particularly how training frequency relates to changes in cognitive function (including cognitive decline).

 

 

 

Project 7: Longitudinal Investigations of Cognitive Training in Adults with Depression and Mild Cognitive Impairments

Status: Ethics approved; data collection ongoing

Required Sample: ~100 participants

Assessments: Initial 2h per participant = 200h (at home); 3 x 1h one-on-one follow-up in USYD labs = 300hrs

AM Training: 15-20 mins, 5 x per week, for 4 weeks (at home)

Team:    Damian Birney, Shannon Web (PhD Student)

Summary: This study will examine the outcomes of cognitive training in participants who have clinical level depression and/or cognitive impairments. This will involve regular cognitive assessments and face-to-face neurological examinations. This study will assist us to examine the potential benefit of cognitive training in clinical populations and the viability of large-scale low cost interventions for treating severe cognitive illnesses such as dementia.

 

 

Project 8: Can mood facilitate cognitive training in adults with depression and mild cognitive impairments?

Status: Ethics application submitted

Required Sample: ~100 participants

Assessments: Initial 2h per participant; 3 x 1h follow-ups in USYD labs = 600hrs

Cognitive Training: 15-20 mins, 5 x per week, for 4 weeks (at home)

Team:    Shannon Web (PhD student), Damian Birney, Vanessa Loh

Summary: The research question considers the extent to which mood state—and in particular depressed mood, and associated levels of motivation and engagement—influences performance on cognitive tasks, and as a result attenuates the benefits of a cognitive training program based on those tasks. Mood induction procedures (MIPs) have been shown to have positive effects on depression and motivation. This line of research will provide experimental evidence contributing to our understanding of the causal relationship between mood and the outcome of cognitive interventions.

 

 

Project 9: N-back Cognitive Training in Active Memory

Status: Ongoing

Required Sample: ~200 participants

Assessments: AM assessments (flexibility games)

AM Training: at home as decided by user

Team:    Damian Birney, Vanessa Loh, Kit Double (PhD), Natali Dilevski (Honours)

Summary: The n-back task is a continuous assessment of working memory, inhibition and attention that has been used in clinical settings for over 50 years. The n-back task has previously been associated with substantial gains on trained cognitive functions, which have transferred to non-trained functions (Au et al, 2015). Six variants of this task have been implemented in Active Memory (faces, dual n-back, n-back distraction, flowers, word chain, and synonym chain). The objective of this project is to investigate a representatively sampled group of AM users who have predominately trained on n-back games to compare against those who have trained predominately on non n-back games. A subsample of these participants will be invited to participate in an in-depth follow-up of cognitive and personality assessments.

 

 

 

Project 10: Evolving confidence in the self-regulation of working memory, reasoning, and relational integration on within and between-session training.

Status: Pending (ethics approval received)

Required Sample: ~100 participants

Assessments: 2h per participant = 200h (at home);

AM Training: at home as decided by user

Team:    Damian Birney, Kit Double (PhD)

In this project we continue and extend on earlier investigations (e.g., Birney et al., submitted) of the role of non-cognitive factors (such as personality, motivation, and confidence in one’s correctness of responses) on within- and between-task performance trajectories. The objective of this study is to investigate the non-cognitive correlates of performance trajectories and to explore the similar and different factors that determine engagement over short periods of time (e.g., a single training session) versus longer periods of time (multiple training sessions over time). The analyses draw on the Active Memory survey questions asked at the beginning and end of each training session.

 

 

Project 11: Statistical models for understanding learning trajectories in complex training data.

Status: Ongoing

Required Sample: existing AM data

Assessments:

AM Training: at home as decided by user

Team:    Sally Crips, Robert Kohn, Robert Wood, Damian Birney, Vanessa Loh, David Kohn

 

 

 

Project 12: The effects of personality and metacognition on cognitive training adherence and outcomes

Status: Completed

Required Sample: 831existing AM users

Assessments: Initial 2h per participant = 1700h (at home)

AM Training: at home as decided by user

Team:    Kit Double, Damian Birney, Vanessa Loh

This project examined the effects of personality and metacognitive beliefs on cognitive training adherence and outcomes participants who were younger, more open to experience and had a higher need for cognition stopped training quicker, while, those who were younger, more open to experience and had more malleable views of intelligence performed better

 

 

Project 13: The effects of self-assessment on performance in a cognitive task

Status: Completed

Required Sample: 104 AM Users

Assessments: 2h per participant = 208h (at home)

AM Training: none

Team: Kit Double,Damian Birney

This project examined whether frequent self-assessment influenced performance on a cognitive task. The results suggested that if a participant had high confidence in their abilities then ‘online’ self-assessment was beneficial, whereas if a participant had low confidence then it was detrimental.

 

Project 14: A meta-analysis of the effects of ‘online’ self-assessment on performance.

Status: Completed

Required Sample: existing data

Assessments:

AM Training: none

Team:    Kit Double, Damian Birney

This project conducted a meta-analysis to aggregate the results from existing literature into the benefits of self-assessment during cognitive (memory) tasks. We found that ‘online’ self-assessment was overall beneficial but there was large variation which could be somewhat attributed to difficulty effects. Self-assessment was more beneficial on easy task compared to hard.

 

 

Project 15: The effect ofonline confidence ratings on self-evaluation.

Status: Completed

Required Sample: 88 AM users

Assessments: Initial 30min per participant = 44h (at home)

AM Training: none

Team:    Kit Double,Damian Birney

This project showed that frequently self-assessing (via confidence ratings) oneself resulted in an overall less accurate appraisal of one’s performance. This was largely the result of confidence ratings making confident participants relatively more confident and not confident participants more underconfident.

 

 

Project 16: Cognition-focused interventions for older adults with MCI or dementia: A systematic review of mood and motivation moderators of treatment efficacy.

Status: Completed

Required Sample: N/A

Assessments: 427 studies were identified, 34 were eligible, 6 met full criteria for inclusion.

AM Training: none

Team:    Shannon Webb,Vanessa Loh, Damian Birney

This project conducted a systematic-review of the existing literature into the moderating benefits and impacts of mood and depression in older adults with mild cognitive impairments (MCI). The review aimed to identify relevant data in the field regarding the influence of depression, apathy, and motivational factors on the outcomes of cognitive interventions for older adults with MCI or dementia. Overall, there was limited evidence regarding the direct effects of depression on cognitive outcomes, although there was a trend toward a significant correlation between improved overall neuropsychological symptom severity and improved cognition, and a significant mediation effect, where change in depression severity mediated improvement on subjective memory ratings following cognitive rehabilitation. It was concluded that it could be the case that rather than having a direct influence on cognitive intervention outcomes, depressive symptoms indirectly moderate intervention effects through constructs such as memory awareness.

 

 

Related PhD Projects (on going)

Kit Double (2015 – 2018). Self-regulation and metacognitive processing in cognitive tasks

Shannon Web (2016 – 2019). Cognitive training in adults with depression and mild cognitive impairments

 

Related Honours Projects and Dissertations (completed)

Natali Dilevski (2016). Working memory: Relational Integration and Coordination)

Eva Lo (2016). Spatial Reasoning and Imagery as predictors of complex task performance

Claire Burrows (2016). Stereotype Threat: Obesity and Executive Function

Luke Ashmore-Delaney (2015). Working memory training, goal orientation and learning.

Shenying Bi (2015). Implicit theories: Challenge and feedback-seeking behaviours in spatial rotation training

Joel Bateman (2015). Exploring the components of working memory related to Gf: Integration and coordination

Kara Lisyak (2014). Sense of control as moderators to n-back training outcomes.

 

 

Related Publications and Conferences based on Active Memory research and ARC-DP140101147

 

Currently in preparation

Bateman, J., Birney, D. P., & Loh. V. (in preparation). Exploring f unctions of working memory related to fluid intelligence: Coordination and relational integration.

Bateman, J., Birney, D.P. Validating the Relation Monitoring Task as a predictor of fluid intelligence.

Bateman, J., & Birney, D.P. (submitted). Building to a taxonomy of relational thinking: What connects working memory and fluid reasoning?

Birney, D.P., Ashmore-Delaney, L., & Loh, V. (in preparation). Working memory training, goal orientation and learning.

Birney, D.P., McGrane, J. Howe, A, & Fayn, K. (in preparation). The assessment of learning in adaptive n-back training tasks.

Birney, D.P., Loh, V., & Wood, R.E. (in preparation). Validation of the communication comprehension test as a measure of far transfer following cognitive training.

Birney, D.P., Loh, V., Costa, D., & Gessler, D. (in preparation). The FACT-Cog as an indicator of quality of life in healthy older adults and associations with cognitive training outcomes.

Birney, D.P., Webb, S., Loh, V., Double, K., Lampit, A., Bateman, J. (in preparation). Theoretically derived outcome measures can inform targeted cognitive training: Revised Meta-Analysis of Lampit et al (2014).

Webb, S., Loh, V., & Birney, D.P. (in preparation). Do depressive symptoms and motivational factors (such as levels of apathy and self-efficacy) moderate the outcomes of cognitive training interventions for older adults with MCI/dementia: A systematic review

 

 

Currently under review

Double, K.S., Birney, D.P, Walker, S.A. (submitted). A Meta-analysis and Systematic Review of Reactivity to Judgements of Learning. Memory

Loh, V., & Birney, D. P. (submitted). Cognitive training in the wild: Profiling brain-training users and patterns of play. Health Psychology.

 

 

Published

Birney, D. P., Beckmann, J. F., Beckmann, N., & Double, K. S. (2017). Beyond the intellect: Complexity and learning trajectories in Raven’s Progressive Matrices depend on self-regulatory processes and conative dispositions. Intelligence, 61, 63-77

Birney, D. P. (2015). Challenges for an interdisciplinary consideration of cognitive training. New Directions for Child and Adolescent Development, 147, 21-32.

Double, K. S., & Birney, D. P. (2017). Are you sure about that? Eliciting confidence ratings may influence performance on Raven’s progressive matrices. Thinking & Reasoning, 23(2), 190-206.

Double, K. S., & Birney, D. P. (2016). The effects of personality and metacognitive beliefs on cognitive training adherence and performance. Personality and Individual Differences, 102, 7 – 12.

 

 

Conferences

Double, K.S. & Birney, D. P. (2016). The Role of Metacognition in Cognitive Training Outcomes. Paper presented at the 43rd Australian Experimental Psychology Conference, Melbourne, Australia. 1 April, 2016

Double, K.S. & Birney, D. P. (2016). When do cognitive training programs work? The Role of Metacognition in training and transfer. Paper presented at the Sydney Postgraduate Psychology Conference, Sydney, Australia, 15 November 2015.

Double, K. S. & Birney, D. P. (2016). The Role of Personality in Cognitive Training Adherence and Outcomes. Poster presented at the 8th European Conference on Positive Psychology, Angers, France 28 June 2016.

Double, K.S. & Birney, D. P (2016). Are You Sure About That? Eliciting Confidence Ratings Improves Performance on Raven’s Progressive Matrices. Paper presented at the Sydney Postgraduate Psychology Conference, Sydney, Australia, 17 November 2016.

Birney, D.P., Loh, V., Double, K (2016). Cognitive training in the wild. Next Gen Sydney Brain Training Symposium 2016, University of Sydney, 2 December, 2016.

 

Technical Reports

Wood, R. E., Kinny-Lewis, C. et al. (2016). Active Memory at Work. Unpublished Technical Report: University of Melbourne.

 

 

Prepared by:

Damian Birney, University of Sydney (damian.birney@sydney.edu.au)

16 November, 2016