HomeHardware


Using GPS NMEA data to create navigation apps
Syndetic
To be fair, you're right, nothing serves all purposes, and that's what this site tries to help people with. Sorry if I came across too brash. I was still cooling off from being frustrated with the current state of things. Read below if you're curious about what I mean.

I'm actually amazed that in 2011 no one has come up with a really good system that fits higher-end needs, whether as an all-in-one unit or as a set of hardware + software components built to work perfectly together, without costing a lot. From what I've observed so far, I think the biggest problem is the fact that there's no standardized device protocol that software can easily interface with. If there were, anyone could get data from any GPS device, regardless of breed of device. It would GREATLY lower the barrier to creating software for these things. I write software for a living and have seen good device interfaces and bad ones. The good ones are based on universal standards that anyone can use to get data into and out of a hardware device. As far as I know, all of the GPS receivers and standalone units currently use their own proprietary protocols requiring OS-specific and device-specific drivers as a means to lock people into a particular brand. There's no other reason for it.

So that's my 2 cents, as I see it. If there weren't the aforementioned barrier, there would be MANY more OS-independent software solutions. I'm especially bitten by this since I use OS X and Linux exclusively, and those 2 platforms are the ones that are lacking the most in quality GPS software. In reality the initial fault is with the device manufacturers.
winwaed
Clearly you haven't tried to use the iPad's built-in map program for navigation! Yes the multi-touch interface works well for simple map viewing (as well as a Microsoft Surface, but a bit more portable :-) ), but its navigation and implementation of route-following is a joke. Perhaps the iPad 2 has fixed it, but the iPad 1 version is unusable for road navigation.


Richard
Ken in Regina
Hey Syndetic,

Google "NMEA".

All GPS receivers, including Garmin's, output their data in NEMA. A few of Garmin's models are also capable of being software-switched to the Garmin proprietary protocol, but it's not rocket science to use it because it's a really simple superset of NMEA.

Aside from those few Garmin models, and some older Garmin models that only did Garmin protocol, I'm not aware of any GPS receivers that use a proprietary protocol.

But you may be talking about something different than I am. Perhaps you are refering to the way standalone personal navigation devices (PND), *not* GPS receivers, are built??

I have no idea what protocol the GPS chipsets in smartphones and PNDs use. Or even if there is one. I also don't know what functions are provided on those chipsets and what functions the nav app has to provide.

...ken...
Syndetic
@Ken: I know that the data itself is NMEA, but I was referring not to the data format itself but rather the protocol that you need to use to communicate with devices. Even the bare-bones receivers, like the Globalsat BU-353, for example, require the installation of a kernel-level device driver before you can begin to talk to it. (My Garmin nüvi 1350 is hopeless. Guess I'll have to buy a device that's more "sociable".) The device driver is the "bridge" between the hardware (PND or bare receiver) and the software that humans interact with. It's this bridge layer that gets the raw data from the device and then converts it to NMEA. Just checked: according to Wikipedia, even NMEA is proprietary, although it's been reversed-engineered enough that at the very least coordinate data is trivial to fetch. I'll keep researching this to see what I can find out there. I'd really like to be able to set my laptop's current location so I can test out the new HTML5 Geolocation API. I also want to create something that'll send my coordinates to a server, pull Google Maps data, and then overlay my location onto it so I can send people to a web page and let them see where I'm at. (Yes, I'm aware of GPSGate now, but hey, I'm a hacker, I wanna write my own stuff! .)
tcassidy
What you need is a GPS with a serial connection. Then, no conversion would be necessary. I think you can still get the Garmin GPS18 in that format.

Terry
Marvin Hlavac
... or, if you like the GlobalSat BU-353 you mentioned earlier, you could buy identical unit with a serial connector (instead of USB). Search for GlobalSat BR-355.
Ken in Regina
Hi Syndetic,

If you want to hack your own code for this stuff you've still got some homework to do. But the good thing is that we're here to help you.

The Role of the Driver:

With respect, you're wrong about the role of the driver that you have to install. The only reason you have to install the driver on your computer is because your navigation software only knows how to communicate with a COM port. In 2011!!!

So if you buy a GPS receiver with a USB connector you must install a "driver" that is actually a USB-to-COM port converter. All it does is make the [real] USB port look like a [virtual] COM port so your navigation software can communicate with the GPS receiver. It has no other purpose.

Some programs do not require this. Garmin's Mobile PC has the ability to detect and use a USB GPS receiver directly but it's one of the few nav programs that can.

As Terry and Marvin have suggested, you can buy a GPS receiver with a serial connector instead of a USB connector and then you would not require the USB-to-COM port driver. Of course you would need a computer with an actual hardware COM port. ... Do they even make laptops/netbooks with COM ports any more?

NMEA Data:

The GPS receivers, like the BU-353 or BR-355 are sending pure NMEA data on the cable, regardless of which type of connector. That is, that type of GPS receiver is a complete receiver. It has an antenna that receives the GPS satellite signals and a DSP (digital signal processor) that analyzes the incoming satellite data, figures out the position (and a bunch of other things) and sends it over the cable in standard NMEA sentences using a UART that transmits it as standard ASCII serial data.

There is no further conversion of the data required. Nav apps can use the relevant NMEA sentences exactly as they receive them.

I don't know where you saw that the NMEA standard is proprietary. It's sure not on any of the WikiPedia articles I've seen regarding NMEA 0183.

NMEA stands for National Marine Electronics Association. They have a number of standards. The one we are specifically refering to is NMEA's standard number 0183. This is a combined electrical and data specification for communication between marine electronic devices, including GPS devices. Specifically, "The NMEA 0183 standard uses a simple ASCII, serial communications protocol that defines how data is transmitted in a 'sentence' from one 'talker' to multiple 'listeners' at a time." It is an open standard that is available to anyone to use.

GPS Receiver? Or something else?

Your Nuvi 1350 is not a GPS receiver. It's a complete standalone personal navigation device (PND) that contains a GPS receiver and a complete set of navigation functions. And a bunch of other stuff that is handy but has little or nothing to do with navigation (picture viewer, calculator, etc.). It's a complete computer system that happens to contain a GPS receiver.

For the purpose of using it as a GPS receiver for other navigation software, it might be hopeless. There are some Nuvi models that will output NMEA data to the USB port if you know the correct ritual. But that is not their purpose and most Nuvi models don't.

If you want a self-contained PND that is also easy to use as a GPS receiver for a PC, you would be better served with one of Garmin's handhelds, like the eTrex models, Oregon models, etc.

But the least expensive and most effective way to get a GPS receiver for use with navigation software on a PC is to just buy one that's designed for that purpose. Then you can hack away to your heart's content.

I hope that helps.

In fairness I should probably mention that my background is more than thirty years as an IT professional in the telecommunications industry. I've done, designed and managed software development and have also had a significant period of time dealing with telecom data protocols of all sorts, including some time working on IEEE standards working groups. I'm retired now but I still like to show off a little when the opportunity presents itself.

And if I'm wrong on any of this I'll be quite happy to apologize and learn something from it.

...ken...
Syndetic
@Ken: Yes, I've still got quite a bit of homework left to do.

I'm a GPS virgin who just started down this DIY route only a few days ago, even though I've had a GPS for longer (okay, 8 months). As for the device driver: A-HA! I didn't know it was to create a virtual COM port. Now it makes a lot of sense. I really did think that it was because of proprietary device protocols (handshaking, data packets, etc.), since it didn't make sense otherwise. In all of the places I've looked online in the past few days, no one has actually said what you said. But thanks for saying it, otherwise I would have gone on a long goose chase. I think my best bet, based on what you've said, is to get a bare receiver like the BU-353. My laptop doesn't have any serial ports, so I'll get the USB version. Given your background (yep, impressive; if you've got it, flaunt it ), now I can talk more geek. So if I were to start completely from scratch (which I don't plan to do, but I'm curious anyway), would I just open up a channel to the proper USB device and immediately start exchanging NMEA data "sentences" with it? If it's really that simple, then wow, the sky's the limit. Just need a Mi-Fi or something similar so I can push the data out to a remote server while on the road.

Ken, thanks for all the info. You didn't have to write all that, so I appreciate it. You've saved me days of wandering the internet in search of a ghost!

Michael
Ken in Regina
I'm totally unfamiliar with USB device drivers and how to interact with them (I've done lots of driver code but USB is after my time) but, yes, if you know how, you can just set up an app to find which USB port has the necessary data coming in on it, grab the NMEA sentences and do what you wish with them.

Or you can just use the driver that comes with the BU-353 (or whatever GPS puck you decide to get) and grab the data from the virtual COM port it creates. That's what I would do. But I'm old school.

There are lots of articles on the internet that describe the NMEA sentences - their content and composition. So once you have a connection to the port, it should be easy to snag the ones you want and use them.

If you've got the chops for doing the USB coms or decide to use the virtual COM port then your next step is to figure out which sentences you need for location.

For what it's worth, you don't need to "exchange" anything with the receiver. As the definition of NMEA 0183 states, the standard describes the communication for one "talker" and multiple "listeners". It's predominantly a one-way exercise: the GPS receiver pumps out location (and other) data and the "listener" does stuff with the data.

If I understand you correctly, you want to construct a simple "listener" app that will just pick off the sentences that tell you the location and do something with them.

The only reason you would add a "talk-back" function to send data *to* a GPS receiver is if the receiver has multiple options (e.g. COM port speed, multiple "refresh" rates) and you want to be able to set/reset them.

One thing you need to know is that the receiver sends the sentences in a particular order, essentially a virtual "packet" if you want to think about it that way, and then starts over again in a repeating cycle. The typical cycle frequency is 1Hz, e.g. one complete set of sentences every second.

The simplest way to get a handle on this, since it's just ASCII serial data, is to install the driver that comes with the receiver, connect the receiver, fire up a terminal/telnet app and watch the data come in. It will be instantly obvious what's happening. Capture it to a file and do a little greping, or whatever the modern *nix equivalent is, to fish out the sentences you want.

There are Windows apps (free) that will do all that (display and capture the raw NMEA data) for you as well as using the data to display the current satellite constellation and the horizontal and vertical position error (dilution of precision). Etc.

There's probably no great gratification in getting to the point where you can see, capture and manipulate the raw data. That's duck soup. The fun stuff starts once you've got the data and start figuring what you want to do with it. That's the complicated part.

...ken...
Syndetic
I'm not familiar with USB programming either. I just looked into it a bit and found a library which lets you talk to a USB device by using its vendor ID + device ID (there's a huge list of all known vendor and device IDs out there). Then you can immediately get an input/output stream pair for the device and begin chatting. I previously used the term "exchange" because I had assumed that in order to get the GPS to start sending data, you had to first tell it, "hey, give me data of type X". Apparently that's not true from what you're saying; it constantly spews NMEA sentences over the bus all the time (?). So if it sends me an ASCII stream once a second, I assume I'd have to wait for the right cycle to get the data I'm looking for since there are many different pieces of data, and it won't send it all in one packet or frame. So I'd have to wait my turn to get the sentence containing lat/lon coordinates. You're right, not much gratification with just getting a raw NMEA dump. I'd like to put up some kind of web-based mapping tool to track my location in real-time. I'd also like to experiment with setting my machine's clock (I *think* GPSes receive atomic clock time) and location so that when location-based services ask my OS where I'm at it can reply with something accurate, and when I'm disconnected from the internet I have a reliable time source. The web-based thing isn't complicated, but the integration with the OS might be; I may have to dig deep for that. If you've got any other interesting ideas for what else to do with the data, I'm all ears.
BillLee
If you are working in a Linux world (and I suspect Mac OS as well), the only sensible way is to use the gpsd daemon. I cannot imagine anyone really wanting to try and replicate what that capabilty already supplies by worrying about device drivers for every type of hardware out there.

I did just exactly that for my home-grown gps tracking system, using Google maps, and FireFox, etc. Yes, FireFox was lacking but that was fixed with a small local mod that hopefully will be in FF4 (whenever I get around to trying it! ) And, no, the application is not available to the public yet but will be soon in order to satisfy Google's terms of service.

Regards,

Bill Lee
MrUmbra
When you gain access to the NMEA data it will look like this:

$GPRMC,020648.000,A,4103.589579,N,07520.688383,W,0.03,293.66,220411,,,A*7C
$GPGGA,020649.000,4103.589577,N,07520.688386,W,1,9,0.87,423.876,M,-33.891,M,,*6C
$GPGSA,A,3,24,13,07,08,06,03,11,28,19,,,,1.72,0.87,1.49*09
$GPGSV,3,1,11,07,76,287,37,19,65,056,35,08,44,313,34,11,39,160,37*7D
$GPGSV,3,2,11,03,32,053,34,13,22,210,24,06,21,051,22,28,20,282,24*7B
$GPGSV,3,3,11,24,20,158,32,16,09,086,,26,06,330,*42

These are the most common sentences. You can search the web for what they mean.

GPRMC provides contains GPS recommended minimum fix data :

$ Sentence Init
Sentence ID
Time of Fix HHMMSS.sss (UTC)
Fix validity flag A/V
Latitude DDMM.mmm, N/S
Longitude DDDMM.mmm, E/W
Speed Over Ground knots
Course Over Ground Degrees True
Date of Fix DDMMYY
MagVar, E/W Degrees (Empty fields in this example)
FAA Validy Flag A/V
* (End of data marker)
hh Check sum Hex
<CR><LF>

NMEA defines many data sentences. They legally defend the actual spec documents very aggressively .. you need to buy them

--- CHAS
Syndetic
@BillLee: I plan to use gpsd in some capacity, or maybe even fully. We'll see once I get my receiver in the mail and I start playing with it. But my curiosity is inspiring me to do some port banging, too, just to get a closer look at what's going on.

You should most definitely move to FF4 and sample all the new HTML5/CSS3 fruit. It's amazing what you can do with it. And if you can somehow justify creating an HTML5-only site and leaving IE behind, the web instantly gets very easy and fun — yes, fun! — to program for. I recommend trying out the new HTML5 Geolocation API which lets you track a client in real-time. Set a repeating timer, grab the lat/lon, position a Google Maps marker at the location, and voila. Let us all know when your app is finished and approved.

@MrUmbra: Thanks. Oy, I guess they need to recoup the costs of engineering the spec.
Ken in Regina
Syndetic, it's not about recouping the costs. In one of my former lives I was directly involved in [telecom and project management] standards work. The issue is not one of cost recovery. The issue is controlling the standard so it doesn't go off in a gazillion different - usually proprietary - directions. That's one of the main things a standards body is trying to protect against: all the vendors/manufacturers who want to get an "edge" by taking some or all of the standard in a proprietary direction that favours their product(s). To guard against that you must be prepared to defend the standard(s) vigorously.

That does not mean you stop people from using them. It means you stop people from changing them arbitrarily to suit their own purposes. And to clarify that last point, anyone can change them in their products, too. They just can't call themselves NMEA 0183 compliant any longer, which pretty much takes their products right out of the mainstream market.

...ken...
Syndetic
Ken: I'm totally in support of standards. Most of what I do day-to-day is dependent upon the existence of and adherence to standards, without which I'd be ripping my hair out all the time. What I was referring to specifically was the fact that they charge people money to be able to read the standards docs.
© laptopgpsworld.com About