there could be many reasons why this is happening ( server side issues)
1) For a start ensure that you dont have (Process Resource Monitor) installed on your server. This is a tool that basically monitors cpu/resource usage on your server and will disable mencoder if its using too much cpu. If you dont want to uninstall this application then ensure that you have correctly added mencoder to the exclude list.
2) Make sure that you have enough time set for your max_execution_time in your php.ini file ( try something like 300)
