The short version
Soy gets accused of feminizing men and fueling breast cancer, and both charges rest on a misunderstanding of one molecule. Soy's isoflavones are often called plant estrogens, but that label oversells the resemblance. A peer-reviewed review lays out what they actually are.
Primary source 01 / 04
Isoflavones are classified as both phytoestrogens and selective estrogen receptor modulators.
A selective estrogen receptor modulator behaves differently from the body's own estrogen, tissue by tissue, which is why the blanket estrogen framing misleads. On the specific male-hormone fear, the review is blunt.
Primary source 02 / 04
neither soy intake nor isoflavone exposure affects testosterone levels in men, effects on this hormone are not a possible explanation for the proposed protective effects of soy against prostate cancer [283].
The breast-cancer worry, reversed
The other fear runs the opposite direction from the evidence. In the populations that eat the most soy, breast-cancer rates are not elevated, and the research on soy and breast cancer has largely reassured rather than alarmed. The molecule that started the scare, it turns out, is chemically related to a class of drugs used to treat both breast cancer and osteoporosis. The plausible worry became, on inspection, a non-worry.
The federal fact sheet
The NIH keeps a plain-language fact sheet on soy. Its summary of the safety question runs three sentences.
Primary source 03 / 04
Many studies have been conducted on the safety and usefulness of soy products. We know enough to conclude that there are potential beneficial effects on blood cholesterol levels, menopausal hot flashes, breast cancer risk, bone density, and blood pressure. However, effects may vary for different types of soy products and in different populations.
On breast cancer specifically, the same sheet reads in the same direction as the review evidence above.
Primary source 04 / 04
Observational studies indicate that among Asian and Asian-American women, higher dietary intakes of soy are associated with a lower risk of breast cancer.