Annoying. If you need to delete all of the RSS feeds setup on your IE7, you will have to select each entry, right click, select delete, select OK. A quicker way is to go straight to the folder:
C:\Users\awing\AppData\Local\Microsoft\Feeds
and delete them from there.