OpenStreetMap

I just finished the Maine address import and thought I might share a narrative.

Origin Story

In early 2019, I was driving home, getting directions from google maps. I had grown apathetic to fast food restaurants highlighted on the map, but on this trip, the google lady vomited forth: “Turn left at People’s United Bank”. She might have said “turn left on Main Street” or just “turn left” but the application I trusted for navigation went out of its way to shill a business. I was left wondering “what else does it do”. Could advertisers pay to /optimize/ my route, sending me past their venue? That sounds /crazy/, and plausible. So it was time to make a change.

The alternative I found was OsmAnd. But upon trying it, I found Maine was missing nearly all addresses. I was baffled how that could be, and dismayed that I might crawl back to google maps. This wasn’t just impeding me, this would block anyone in Maine from switching, and /I am stubborn/. I realize everyone can contribute to OSM and Maine had a handful of contributors, but at the current rate, Maine may not be navigable within my lifetime. It needed a push.

Programming Montage

I had a vague idea of “get data from here” and “put data there”. The APIs had documentation and I had a compiler, how hard could this be? I started developing tools in July of 2019. In the beginning, I had two concerns:

• I could lose interest and leave the work “half done”, and regret that I ever started
• I could do a terrible job… and regret that I ever started

Looking back, this project held my attention the whole way and I’m satisfied with the results.

Side node: Did I look for existing tools? Yes, but I’m a software engineer and when you have a hammer everything looks like a nail.

Using my language of choice, I wrote an Osm API client, a client for Maine’s address API, a E911-to-OSM schema translator, a conflation engine, and a command line interface to tie it together.

Lessons Learned

Some effects of writing my own tools:

• Good: I am now intimately acquainted with the many challenges of each component
• Good: I had complete control and visibility into every part of the process
• Bad: I never polished the user interface, so it was only realistically usable by me. I couldn’t divide the work between other mappers
• Good: I had hoped the tools would be useful to the community. Some parts are already being used in other projects, some parts are effectively dead
• Good: I was able to contribute fixes to other, related projects
• Good: I have a substantial portfolio of work to show potential employers

With passable tools, I started processing the import in January of 2020. My tools evolved constantly throughout the import handling ever-weirder corner cases. Her are a few of the poor assumptions I made along the way:

• The source data will be accurate and clean
• The state won’t change their API and schema in the middle of my project
• Geometry is easy, simulating the human decision making process for picking things to combine is easy
• House numbers always relate to roads
• There wouldn’t be odd bunches of addresses that need to be handled specially by context (trailer parks and apartment buildings)
• There would never be ambiguity about which municipality something is in, or what a municipality is called (villages, unorganized territories, reservations, plantations)
• There won’t be many data conflicts to manually resolve (Portland took ~20 hours, Sanford took ~30 hours)
• I can use existing OSM road data to validate addresses
• I can use existing OSM municipal boundaries to validate addresses
• Names will be in canonical form (Rose Road, Roses Road, Rose’s Road, Roses’ Road, ROSES ROAD, Roses Rd, Rosesroad, Roses Street, etc)
• And many more…

Personal Impact

I was working at a large international company, doing work that I genuinely believed in (we literally saved lives, a few a day) but the work had turned gray. I didn’t notice I was burned out until I found this project, and absolutely loved working on it. It reminded me that programming /is/ fun, and I started to realize work had /stopped/ being fun. There were many reasons, but primarily, I wasn’t learning anything new. I spoke to my manager and asked for a whole lot of unpaid time off, which was declined. So I quit (and got my unpaid time off). Aside from other interests, I spent the next year working this project. It has given me a great sense of accomplishment and contribution. I met many characters long the way and learned a lot about my state. And finally, I can use OsmAnd for navigation.

Did I mention I’m available for hire? I offer this project as my resume while I seek GIS opportunities.

-Alex

What’s Next

I’m planning Maine roads, or getting a job, whichever comes first.