According to your system configuration you can use Windows Server 2008 x64. The latest version is 6001.18000.080118-1840_amd64fre_Server_en-us-KRMSXFRE_EN_DVD.iso and you can download it from here.
It is possible that the installation “hangs” for a few minutes, however if after 10 minutes it is still not responding, something went wrong… In that case search the Microsoft Technet Setup Deployment forums to find more information about your problem. If no similar problems are posted you can add a new post with your problem and the Microsoft Technet professionals will help you out! 🙂