yes, and then you use screws that came with the case, the manual explains which screws you need, the black stand off is there to help you and should see it when the mobo is in the right place.
And no a heat sink to any thing should not be loose as it would not be able to do what it's there for, how ever it should not be to tight either.
View attachment 244673
If you have taken it to the shop and they say it's working as it should be why it not working for you ?, did they show it all working or did you just take there word for it ?.
Did they check the on-board as well as the GPU ?.
You said you had the on-board video output yes ?, can you supply a picture with the video card connected as you were on about it not sitting right in the slot.