Beat-to-beat oscillations in heart-rate and systolic arterial-pressure are known to be modulated by the autonomic nervous system. Their characteristics are used in psychophysiology to assess the autonomic reaction to psychological/cognitive challenges (PC). However, factors other than autonomic modulation may strongly influence the estimation of these cardiovascular indexes (CI). Here, two such confounding factors were investigated in detail. First, the between-task and inter-individual differences in respiratory patterns, especially in PC involving speech were found to be strongly reflected in the CI. The second factor is the presence of within-task dynamics in the CI reaction to PC. Since these two factors are intrinsic in the reaction to PC, they can have a profound impact on the indirect estimates of autonomic reaction through CI. Controlling them during PC may be difficult. However, their effects should be minimized, for example by avoiding PC involving speech and choosing appropriate data epochs for the analysis. These effects must also be considered in the physiological interpretation of CI, in order to ensure their usefulness in future research.