Dear Dr Zalesky,
I would like to run a 2x3 mixed ANOVA, and, after reading all the posts here on the Forum, I tried to build appropriate design matrices and contrats.
I have 3 groups (A, B and C; between-subject variable) tested at two time-points (W and N2, within-subject variable) and I would like to test the interaction between GROUP and TIME, the main effect of TIME and the main effect of GROUP. Then, if any significant interaction or main effect is found, I would like to carry on the post-hoc tests. The design matrices and contrasts are reported in the Excel file and I was wondering if they are correct or not:
- design matrix and contrasts used to test interactions and main effects (sheet “2wayMixedANOVA”)
- design matrix and contrasts for the post-hoc tests in case of a significant main effect of the GROUP (sheet “Post_hoc_for_MainEffectGROUP”) --> independent t-tests
- design matrix and contrasts for the post-hoc tests in case of a significant main effect of the TIME (sheet “Post_hoc_for_MainEffectTIME”) --> paired t-tests (repeated measures)
I also have another question regarding the exchange blocks. If I have understood correctly, they should be used when there are within-subjects variables (repeated measures), so in my case they should be used when testing the interaction/main effects and when doing post-hoc tests following a significant main effect of the variable TIME (a within-subjects variable).
I hope my explanation is sufficiently clear and apologies for the long message!
Thanks a lot,
Best wishes
Isotta
Hi Isotta,
When testing for the interaction effects, the two columns corresponding to the main effects of group should be ommitted from the design matrix (2nd and 3rd column in your Excel file). If they are not omitted, a rank deficent warning will appear. This issue is not specific to the NBS, but a general property of least squares regression. Other than that, the design matrix and contrasts for the interaction effect look correct.
For the main effect of group, we don't need two time points. To test for the average effect of group across time, you could average your measures across the two time points for each subject and perform inference on the average.
The posthoc tests appear to be correct.
Your understanding of exchange blocks is also correct - they should only be used for within-subject effects, which in your case is whenever time is modelled in the design matrix.
Let me know if you have any further questions,
andrew
Originally posted by Isotta Rigoni:
Dear Dr Zalesky,
I would like to run a 2x3 mixed ANOVA, and, after reading all the posts here on the Forum, I tried to build appropriate design matrices and contrats.
I have 3 groups (A, B and C; between-subject variable) tested at two time-points (W and N2, within-subject variable) and I would like to test the interaction between GROUP and TIME, the main effect of TIME and the main effect of GROUP. Then, if any significant interaction or main effect is found, I would like to carry on the post-hoc tests. The design matrices and contrasts are reported in the Excel file and I was wondering if they are correct or not:
- design matrix and contrasts used to test interactions and main effects (sheet “2wayMixedANOVA”)
- design matrix and contrasts for the post-hoc tests in case of a significant main effect of the GROUP (sheet “Post_hoc_for_MainEffectGROUP”) --> independent t-tests
- design matrix and contrasts for the post-hoc tests in case of a significant main effect of the TIME (sheet “Post_hoc_for_MainEffectTIME”) --> paired t-tests (repeated measures)
I also have another question regarding the exchange blocks. If I have understood correctly, they should be used when there are within-subjects variables (repeated measures), so in my case they should be used when testing the interaction/main effects and when doing post-hoc tests following a significant main effect of the variable TIME (a within-subjects variable).
I hope my explanation is sufficiently clear and apologies for the long message!
Thanks a lot,
Best wishes
Isotta
Dear Dr Zalesky,
thanks for the clear reply!
I have tested the interaction effects with the original design matrix (including the 2nd and 3rd columns) and no 'rank-deficient' warning message appears. The only warning message I get is 'Warning: The EraseMode property is no longer supported and will error in a future release' but it's not linked to the matrix being rank-deficient. So I am not sure if I should still use this design matrix and contrasts to test the interacion or if I should indeed remove the 2nd and 3rd columns when testing the interaction.
Regarding the testing of the group effect, I see your point but is what I am doing incorrect or can I keep it this way?
Thanks again for the help,
Best wishes,
Isotta
Ignore the EraseMode warning. It is irrelvant and relates to a version change.
I'm surprised why including the 2nd and 3rd columns does not give a rank deficieny warning. Are your results the same when you remove these two columns?
Sure - you can test the group effect as you like.
andrew
Originally posted by Isotta Rigoni:
Dear Dr Zalesky,
thanks for the clear reply!
I have tested the interaction effects with the original design matrix (including the 2nd and 3rd columns) and no 'rank-deficient' warning message appears. The only warning message I get is 'Warning: The EraseMode property is no longer supported and will error in a future release' but it's not linked to the matrix being rank-deficient. So I am not sure if I should still use this design matrix and contrasts to test the interacion or if I should indeed remove the 2nd and 3rd columns when testing the interaction.
Regarding the testing of the group effect, I see your point but is what I am doing incorrect or can I keep it this way?
Thanks again for the help,
Best wishes,
Isotta
I have re-run the interaction (C1) on the same data after removing the 2nd and 3rd columns and changing the contrast accordingly (C1=[0 1 0 zeros(1,num_subjects)]). As for the original test (with full design-matrix and original C1=[0 0 0 1 0 zeros(1,num_subjects)]), I get non-significant results and a very similar "lowest p-value" (p=0.22 and p=0.24).
I understand that this is not the best way of testing if the results are consistent, but that's all I can do given my data.
In both of the cases, I don't get any Warning message concerning rank-deficiency matrix.
I might be doing something else wrong without me realizing though...
Hi Issa,
given that your results are consistent between the two approaches, perhaps this is something that won't have any practical consequences. I.e. your conclusions will be the same irrespective of the approach.
One other thing to note is that testing multiple interaction effects usually requires larger sample sizes to acehive adequate power, compared to simple designs including only one interaction or a few main effects. If your sample is small/modest, you might want to consider simplifying your design, otherwise you might not have adequate statistical power to detect complex interaction effects.
andrew
Originally posted by Isotta Rigoni:
I have re-run the interaction (C1) on the same data after removing the 2nd and 3rd columns and changing the contrast accordingly (C1=[0 1 0 zeros(1,num_subjects)]). As for the original test (with full design-matrix and original C1=[0 0 0 1 0 zeros(1,num_subjects)]), I get non-significant results and a very similar "lowest p-value" (p=0.22 and p=0.24).
I understand that this is not the best way of testing if the results are consistent, but that's all I can do given my data.
In both of the cases, I don't get any Warning message concerning rank-deficiency matrix.
I might be doing something else wrong without me realizing though...
Dear Andrew,
yes a modest sample size could be a problem here as I only have 30 subjects per group (90 subjects in total), for each of whom we have two measurements in time (in N2 and in W).
When you say that I could simplify the design, you are referring, for example to reducing the number of groups to only two (so that there will only be one interaction to test)? Or there's any other way to simplify the desing that I cannot think of right now?
Ok, that's it...I have already exploited your kindness too much...I promise I won’t bother anymore after this 😊
Thanks a lot for the help and have a nice day!
Isotta