Media can be only accessed when logged in?

Hi,

I have a bit strange problem. My site recently went to SSL. I made all changes (that I found via google) and got it working except images and media cannot be accessed.

The links are properly in https form in the html but those doesn't work until I log in to wp admin interface and test site from there.

I guess that there's some setting that I have completely missed bu after a lot of searching, I didn't found anything that would fix this.

Any ideas?

Theme is twenty-one and used plugin is Really Simple SSL.