There are many things that can be causing this.
1. Your GPS is a consumer device. It is only accurate to within a few meters at the very best. And that depends on having a very good signal. Your GPS may have a screen that tells you what it thinks the accuracy of the current reading is. That is useful information but it is also only an estimate and does not -- can not -- tell you which direction it is inaccurate.
.. 1a. You can improve this by standing in one place for at least a couple of minutes to let the reading stabilize before recording the coordinates. If your device has an "averaging" feature, use it for a minimum of thirty seconds, preferably more, whenever recording a location.
For more information on GPS "accuracy" you can read this thread: http://www.laptopgpsworld.com/777-what-gps-accuracy-anyway
2. It is necessary to make sure your GPS device and Google Maps/Earth are using the same "datum" (look for "geodetic datum
" with your search engine). The simplest is to make sure your GPS device is set to WGS84 (there is usually a place in the Settings on your device to do this). I'm pretty sure WGS84 is the datum used by Google Maps. Using different datums on the two displays will result in huge differences in where the same set of coordinates is displayed on the maps.
3. The calibration of Google Maps/Google Earth is not always correct. You can see this on the maps if you have set the view to the combined road map and aerial view. In many places you will see the roads from the road map do not lay exactly on top of the roads in the aerial view. I have seen examples where the calibration is out by as much as 1/2 mile. Yes, that means the road from the road map was laying over the aerial map view 1/2 mile from where the same road was displayed in the aerial view.
4. You did not say which Google Maps view you are comparing with the maps on your GPS device.When you combine the inaccuracy of your consumer GPS device (point #1) and the inaccuracy of Google Maps calibration (point #3) you will find that the opportunity for accumulated error can be very large.
5. Similar to point #3, the calibration of the maps in your GPS device probably is not perfect, either. If the calibration error of the maps in your device is in the opposite direction of the calibration error in Google Maps you can see the opportunity for another large error in what you see displayed on your device maps versus Google Maps even if the calibration error in each is small.
There are other considerations but those are the major ones to be aware of.