Updating Drupal Core always is a litte bit hardcore.
You need first delete most of the old code. Here is a step by step instruction for the update process.
It is assumed that your custom modules and templates are in the sites/all folder.
Otherwise you get some more work to save the files. If you want to preserve your .htaccess robots.txt and favicon.ico save it before updating.
1. Backup of your Drupal instance.
2. Download the latest release of your current Drupal version.
3. Extract the [tar ball or zip] Drupal package.
4. Set your site on maintenance mode
5. Delete all the files & folders inside your original Drupal instance except for /sites folder and any custom files you added elsewhere.
6. Copy all the folders and files except /sites from inside the extracted Drupal package [tar ball or zip package] into your original Drupal instance.
7. If the update release includes changes to settings.php replace old settings.php in .../sites/default/ with the new one, and edit site-specific entries (eg database name, user, and password)
8. If you modified files such as .htaccess or robots.txt re-apply those changes to the new files.
9. Login to your site as administrator or as user no 1
10. Run update.php by navigating to http://...yoursitename/update.php
11. Follow the process to update your Drupal instance
12. Disable maintenance mode