Low birth weight (LBW; < 2,500 g) has been identified as a risk factor for adverse mental health outcomes over the life span. However, little is known about the association of LBW and suicidal ideation in middle and late adulthood. We investigated N = 8,278 participants of a representative community cohort: 3,849 men (46.5%) and 4,429 women (53.5%) (35–74 years of age). We assessed standardized measures of mental distress, sociodemographics, health behavior, and somatic factors (based on an extensive medical assessment). Controlling for these confounders, we examined the relationship of birth weight and suicidal ideation in logistic regression models. As men and women differ with regard to their susceptibility to suicidal ideation and behavior, we tested sex-dependent effects. LBW was reported by 458 participants (5.5%). In men, LBW was associated with a higher likelihood of reporting suicidal ideation (OR 2.92, 95% CI 1.58–5.12). In women, there was no such relationship. The findings underscore the interrelatedness of the physical and psychological domain, the role of early adversity in suicidal ideation, and they identify a vulnerable group whose numbers are expected to grow. They also indicate other risk factors for suicidal ideation in the community (mental distress, lack of social support, and health risk behavior).