OpenStreetMap
I think I have mentioned before that the Telenav OSM team is always thinking of more ways to help improve OpenStreetMap. Something we have recently started looking into is supplying proposed changes directly as JOSM compatible XML files. This way, we could propose changes and additions to the map in chunks, instead of one by one like we have done for MapRoulette thus far.
So looking into the JOSM file format, we created a small (hypothetical) test file that just adds some ways that connect to the current network. Shown below is the process of merging that test file with existing OSM data in three stages: Base OSM layer, change layer overlaid, and finally the merged result.
Did you just notice something STRANGE here? When I made the change layer visible, some but not all the new ways appeared on the map - but after merging, the ways all appear. What makes this even weirder is that I can select the invisible way in the change layer, see and change tags etc. Is this a JOSM bug or is there something about the JOSM file format I do not understand?
If you want to try it for yourself, here is a zip file with both the base OSM data and the proposed change file.
Discussion
Comment from bdhurkett on 9 March 2015 at 09:40
I could replicate the behaviour you described.
At a guess (after ruling out obvious rendering oddities - both visible and invisible ways have the same map style listed in Advanced Info), it’s because the ‘hidden’ ways use a combination of new nodes (with negative ID values, described in changes.osm) and existing nodes (with positive numbers, referenced in changes.osm but only described in past.osm.pbf). Presumably JOSM gives up on rendering it after failing to find a matching node on the same layer. This would explain why some ways display fine before - they only consist of new nodes - and why everything displays fine after the layers are merged.
It makes sense though, I suppose - assuming there’s no implementation problem, the alternatives would be potentially modifying layers that weren’t active, or having mysterious uneditable nodes on the active layer.
One solution could be to include the existing nodes in the changes.osm file - JOSM should merge them with the original automatically since they’d have the same ID, but I don’t know what would happen if the original is modified in the mean-time.
Comment from mvexel on 9 March 2015 at 14:59
Thanks for looking bdhurkett - All existing nodes are actually in the change XML file, so all information needed to reproduce the way are present. And as I mentioned the way is actually there - if you click in the right place, it is selected. We even made sure to order the elements so that there are no forward references in case the contents are read using a SAX-like parser. I did check the map paint / rendering settings, I am using the defaults which I believe should render the indicated ways visibly.
Comment from mvexel on 9 March 2015 at 19:33
I was wrong - there were nodes missing from our file, which explains the missing way display. Still, there is something not quite right about the way JOSM handles these ‘incomplete’ files. I would say either accept them but display a warning that there are missing nodes (there is such a warning in the console output, but not in the UI), or enforce full referential integrity for JXML files.
I also pointed to this entry on the josm-dev mailing list and there are some thoughtful responses there.