Background The reasons for low influenza and pneumococcal vaccine acceptance in the elderly population are largely unknown – despite the great need of vaccines in this risk group. While many studies examine the relationship between factors influencing vaccination, such as sociodemographic characteristics and influenza and pneumococcal vaccination intentions and behavior, psychological factors, such as vaccine-specific attitudes, are underutilized in research on vaccination behaviors and intervention strategies. This article assesses the psychological antecedents of influenza and pneumococcal vaccination in the elderly and assesses the predictive power of psychological vs. sociodemographic and other factors surrounding vaccination, on vaccination behavior. Methods A cross-sectional telephone survey, representative of age, gender and rural/urban residence, was conducted with N = 701 German participants > 60 years of age, during the influenza season of 2016–17. Multiple logistic regressions were conducted to identify the relevant determinants of vaccination behavior. Results Results show unique patterns in the psychological antecedents: while confidence, the belief in the effectiveness of vaccination and calculation, the need for information, complacency, the lack of risk perception and constraints, and perceived practical barriers to vaccination predicted influenza vaccination behavior, only complacency predicted pneumococcal vaccination behavior. The amount of explained variance in influenza vaccination behavior nearly doubles when psychological antecedents of vaccination are taken into account, beyond other factors surrounding vaccination. However, the effect was smaller for pneumococcal vaccination behavior. The results are compared to a subnational sample. Conclusions Understanding the psychological drivers of vaccination can help to plan interventions effectively. Trial registration Deutsches Register Klinische Studien (German Clinical Trials Register) DRKS00012653. Registered 24.11.2017. Retrospectively registered.