By default, GNOME uses the Nautilus file manager, which opens a new window each time you open a folder. You can change this behavior to use one window with Forward and Back buttons.
To change this, double-click on Computer on the desktop, click Edit and then Preferences. You can also select Places > Computer > Edit > Preferences from the menu panel. Click the Behavior tab and click on the box next to the text Always open in browser windows.
To install a program that modifies several aspects of using Nautilus. Install Gtweakui which enables you to modify your GNOME desktop quickly and easily.
Applications > Add/Remove Software then searching for gtweakui will provide the program to install. The program's location can be found under System > Preferences.