Solar energy: better than fossil fuels, worse than anything else.
It is true that most of the impact is caused by the resource depletion -- solar panels use silver, which is quite scarce material. Hopefully, newer technologies will be able to alleviate this problem.
However, even if we discount the resource depletion part of the solar energy impact, our conclusion remain valid. Pollution caused by production process of silicon wafers used in solar panels is significantly higher than the pollution caused by wind power (and orders of magnitude less than pollution from fossil fuels).
Electricity, from coal
Electricity, from solar panels
Electricity, from wind
If you are concerned with environment then, demand hydroelectric power. If not possible, then wind. If not that, solar, and only then settle for anything else.