Just found this page on the Ubuntu website, bookmarking it here. Looks like need to make some changes to get the .htaccess working. Also seems that I could have installed LAMP automatically with this command:
To install the default LAMP stack in Ubuntu 7.04 (Feisty Fawn)
In Feisty, the Ubuntu base system includes Tasksel. You can either install LAMP using tasksel or install the LAMP packages as detailed above.
sudo tasksel install lamp-server
The site also explains how to uninstall if everything goes pearshaped. Hopefully it will not come to that!