OpenStreetMap

alexkemp's diary

Recent diary entries

Beware the Ides of March :: OSMTracker v0.6.11 loses 42% of Photos

Posted by alexkemp on 23 March 2019 in English (English)

Well, it just keeps getting better, doesn’t it?

In the intro I pointed out what had kept me prevaricating & not surveying for 22 months.

Next was the discovery that OSMTracker v0.7 was utter tosh since it would not produce viable tracks on my phone.

Next was that in addition v0.7 also had most GPS missing from the photos, and that diary was a howTo Fix the Photos, putting GPS back into them.

Finally this diary sees me re-install OSMTracker v0.6.11 and go out tracking recently-built houses within Stonebridge Estate on March 21. There were a number of discoveries:–

  1. The Mpow 059 Bluetooth Headphones mostly seemed effective but could not cope with wind-noise (rumble).
     
    It blows a lot in Britain (one of the windiest places in the world) and low-pitched rumble from microphones is a deal-breaker if you work outdoors. The headphones were extremely comfortable, but the microphone was a teeny little hole with no easy means to apply a dead kitten to it. The wind was actually very modest, but there were occasions when I could not hear what I was saying. I’ve returned them to Amazon, and have ordered a Jabra BlueParrott B450-XT Headset to replace them: Jabra BlueParrott B450-XT Headset  
  2. After discovering that v0.7 is actually OSM-does-not-track I checked out v0.6.11 more carefully. It is losing 42% of all photos.
      
    OSMTracker maintains a separate database of photos + voice-notes + GPS track (“track1”). The android smartphone also saves those photos within the standard camera location (2x). Having finished a track (or before) it is possible to “Export a GPX track” which also places JPG + 3GPP + GPX files within their own directory (“osmtracker/2019-03-21_10-34-15”) (now 3x the space). I checked out the number of jpegs saved by osmtracker cf the smartphone:
     
    → OSMTracker: 107 photos
    → Smartphone: 183 photos (42% extra)
     
  3. As best as I can tell OSMTracker is also losing some voice-notes. The problem is that I cannot quantify the extent of the problem.

What a bummer.

Location: St Ann's, City of Nottingham, East Midlands, England, NG3 4QP, United Kingdom

Beware the Ides of March :: Fixing the Photos

Posted by alexkemp on 20 March 2019 in English (English)

After installing OSMTracker v0.7 on my Android SmartPhone on the Ides of March, then using it the next day for the first time, I discovered that not only did it not track (a touch of an existential issue for a GPS Tracker - how come no-one noticed that before?) but photos taken with the App also were lacking almost all GPS info within the metadata, making them useless for Mapillary.

 ~/DCIM/Camera$ identify -verbose IMG_20190316_092818.jpg | fgrep GPS
exif:GPSAltitudeRef: 240/100
exif:GPSInfo: 686

It was a short session and only 43 photos, but I did not want to waste them if it could be helped. Somewhere in my searching I came across two really old plugins for JOSM which worked to fix that absence:

How to Add Geo-tags into Photos within JOSM

My phone is a cheap model from Vodaphone (SmartPrime-7, model VFD 600, Andoid 6.0.1) and has been able to add a full set of GPS tags to the photos that it takes with the built-in utility from the get-go, let alone when using OSMTracker. Even so, after taking photos under OSMTracker-0.7 and exporting a GPX (track-file), even though the gpx contained lat/long/compass for each photo, the photos themselves did not.

Step-by-step

Inside JOSM:–

  1. Press f12 to bring up Preferences
  2. Select Configure available plug-ins (LHS)
  3. (When the selections come up)
    Scroll down/search for ‘photo’
  4. Select photo_geotagging (currently 34867)
    This is what allows you to select & write (via the right-click menu) the GPS position into the exif of that selected file
  5. Select photoadjust (currently 34867)
    This plugin allows imported photos to be positioned & moved upon the JOSM map.
  6. Select Download
  7. Press OK

We now have the ability to load photos on to JOSM as an Image layer and then to place the photos at the place where the photographer was standing, and to register the direction that the camera was facing. Those GPS details can then be placed into the file metadata. This is easy to do but, since it has to be done one-file-at-a-time and cannot be scripted, it is incredibly tedious to do (I only had 43 photos to process and it took me a couple of hours; imagine if there were 43,000 photos!).

This is the step-by-step:–

  1. Inside JOSM press menu: File | Open… (Ctrl+o)
  2. In the Files of Type: select-box choose Image Files (*.jpg)
  3. Now use the Look in: select box and navigate to the directory containing the photo file(s) to import.
    (It is likely that you will want all photos in a particular directory, so select one with the mouse then press Ctrl+a (select all such files) and press OK (it may offer to Correlate images with GPS track - press Cancel))
  4. If necessary, switch ON the Layers window (access via menu: Windows | Layers… (Alt+Shift+l)) which will now show a Geotagged Images layer
  5. In the LHS click on Adjust Photos so the background is blue
  6. There is now a Geotagged Images dialog-box which will show the active photo
    (this should show the first image in the list when you first start; right-click the Geotagged Images layer within the Layers window and choose Jump to next marker (Ctrl+Alt+J) if it does not)
  7. With the photo to process showing in the dialog-box, hold down the Shift key and click at the correct location and a little camera rectangle will appear. Move it to the exact place where that photo was shot.
  8. Hold down the Ctrl key and click in the direction that the camera was facing, and a little arrow will appear to show that.
  9. When satisfied with both camera location for that photo and direction, right-click the Geotagged Images layer within the Layers window and choose Write coordinates to image layer
  10. The first photo is now geo-tagged; 42,999 photos to go.

You will have spotted an option to mass-tag photos using a GPX track but that was NOT available to me since the track had failed as well. Sigh. Well, at least I got these 43 geo-tagged and later uplifted into Mapillary. Then Mapillary vandalised them with unnecessary blurs, again. So I used the blur editor, again, to remove all unnecessary blurs (most of them). And Mapillary completely ignored the fact that I had done this, so none of that work will be enacted, either. Again.

Well, that’s it for me with Mapillary. Does anyone have any other suggestions to replace Mapillary? What is OpenStreetCam like?

Location: St Ann's, City of Nottingham, East Midlands, England, NG3 4QP, United Kingdom

Beware the Ides of March :: OSMTracker v0.7

Posted by alexkemp on 19 March 2019 in English (English)

I’ve made use of OSMTracker v0.6.11 since starting surveying for OSM on 21 March, 2016. It does not do a lot (GPS track, take GPS-located photos + voice notes), but it does them simply, easily & well and — more to the point — that is pretty much all that you need whilst surveying, in my experience.

There is just one thing that I would like to see added to OSMTracker, and that is for the GPS direction (probably GPSImgDirection) to be added to photo’s exif. That would allow Mapillary to know which direction the camera was pointing in when the photo was taken, which would assist it’s processing of the photos after being uploaded.

I found a new version of OSMTracker was available and decided to install it. Boy, was that a mistake.

Early on 16 March was my first survey for 22 months & it went fine. There were one or two odd looks at the chest-harness holding the VanTop Moment 4 camera, but I was mostly unconscious of it & therefore so was everyone else. The idea was to do a half-session then go home which is exactly what I did.

Things went fine on the survey. It was when I got home that the problems emerged.

There were problems uploading the photos to Mapillary from within JOSM, so I switched to a Mapillary webpage & uploaded from there. After uplift Mapillary asked me to locate all pictures using a map since there was zero GPS in the photos’ exif (not entirely true, but nothing that it could use). That is when I began to investigate the problems & eventually make a bug report to osmtracker-android.

Brief Explanation

Before using OSMTracker it is necessary to switch GPS on within your mobile. It should detect if GPS is off & both inform you of that fact & give you the opportunity to switch GPS on when loaded up, but doing it manually before loading OSMTracker avoids all bugs.

The program will drop GPS breadcrumbs once every second by default whilst active. It also takes a photo and/or voice-notes when requested, each of which contains GPS metadata. (There are also a small number of other features which I never use). These items are all auto-stored within a database during use.

When requested, OSMTracker gives an option to export a GPX file. That action exports the photo- + voice- + GPX-files into a named-directory.

A GPX file is a plain-text XML file. The top part of the file contains wpt sections giving GPS-locations + file-links for the Voice recording + Photo files. The bottom part of the file contains a trk section containing trkseg sections for the GPS breadcrumbs. Except that with v0.7 OSMTracker the trk section is empty.

This is what the top & bottom of the buggy GPX look like:–

 <?xml version="1.0" encoding="UTF-8" ?>
 <gpx xmlns="http://www.topografix.com/GPX/1/1" version="1.1" creator="OSMTracker for Android™ - https://github.com/labexp/osmtracker-android" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.topografix.com/GPX/1/1 http://www.topografix.com/GPX/1/1/gpx.xsd ">
    <wpt lat="52.96083105" lon="-1.13161024">
            <ele>106.0</ele>
            <time>2019-03-16T08:12:18Z</time>
            <name><![CDATA[Voice recording]]></name>
            <cmt><![CDATA[compass: 220.21875
                    compass accuracy: 3]]></cmt>
            <link href="2019-03-16_08-12-18.3gpp">
                    <text>2019-03-16_08-12-18.3gpp</text>
            </link>
            <sat>0</sat>
    </wpt>
 …
    <wpt lat="52.9588609" lon="-1.13084253">
            <ele>114.0</ele>
            <time>2019-03-16T09:28:04Z</time>
            <name><![CDATA[Photo]]></name>
            <cmt><![CDATA[compass: 141.21875
                    compass accuracy: 3]]></cmt>
            <link href="2019-03-16_09-28-04.jpg">
                    <text>2019-03-16_09-28-04.jpg</text>
            </link>
            <sat>0</sat>
    </wpt>
    <trk>
            <name><![CDATA[Tracked with OSMTracker for Android™]]></name>
            <trkseg>
            </trkseg>
    </trk>
</gpx>

And that is not all…

The day had at least one other grain of salt to rub into the wound.

On 7 May 2016 I acquired some Sony headcans & placed a dead kitten upon the microphone, as to kill the rumble from the wind. It has been my trusty companion whilst surveying ever since. Ah, no longer, as now the Dead Kitten actually is dead.

The OSMTracker voice recording files all actually existed, but they were empty of sound. All that they had recorded was the sound of silence. My cans were mute. Desolate. Alone Again, Or.

The OSMTracker voice problem has now been confirmed by trying to use it with Sound Recorder, the Android app, and that also records only silence. I’ve ordered some Mpow Bluetooth Headphones, and we shall see if they are unusable outside or not. They are supposed to be arriving Thursday (they arrived today, Wednesday).

Location: St Ann's, City of Nottingham, East Midlands, England, NG3 4QP, United Kingdom

Beware the Ides of March :: Intro

Posted by alexkemp on 18 March 2019 in English (English)

On a day in May 2017 2 crack-heads† tried to kill me as I was mapping. They failed, I dialled 999, the police attended & in the end decided to ignore it (no CCTV, and the police sergeant claiming a lack of resources to investigate).

I guess that it is simple middle-class naivety on my part but I was profoundly affected by what I thought of as the refusal of the police to defend me.

The UK government has taken away the right to bear arms from all except the police, military & criminals (thank you Tony Blair). A former member of the Territorial Army that I know was recently asked to help clear his parent-in-law’s house following her death and discovered viable pistols + shotguns in the attic. He took the pistols back to his house and shortly afterwards was descended upon by the police. I counted 15 police at one stage standing around chatting outside his house + an unknown number inside. The whole palaver went on all night until the next morning, including police film-crews with portable floodlights conducting interviews outside the house with other police. The entire family is now in shock & clearly suffering from PTSD.

I had a touch of PTSD myself for the next 19 months, spending most of my time after the crack-head incident inside my house. I certainly could not face any surveying. In the end it was obvious that I had to drag myself up by the bootstraps and do something — in effect, re-boot myself.

Some little leaflets originally produced by Andy Allan were most useful but I was almost out of them. So, I spent time updating those promotional OSM leaflets and getting them printed.

Whilst huddled in my burrow Mapillary had spent it’s time vandalising my photos, all 6,200 of them. So, I spent time attempting to undo the damage using their blur editor. They ignored all my work.

Finally, there was the old mapping from May 2017 - it was unfinished. So, my new computer needed JOSM installing, etc. and I needed to re-learn how to use JOSM. That process was finally completed on March 14.

I spent the next day — the Ides of March — preparing myself for my re-launch. The obvious place was Stonebridge Park. It was a white patch in 2016 and, whilst Bing has still not updated to show the houses that were built there years ago, other imagery is now available which does.

I had acquired a cheap GoPro-alternative + chest-harness (it shoots in upto 4k, which is remarkable) to add to my mobile for recording stuff. I also had sourced & installed an update for OSMTracker. Hooray! We were good to go the next day. What could possibly go wrong?

†I have zero proof that they were crack-heads. I call them that because they were behaving like Tigger (“Bouncing is what Tiggers do best”) and, after thinking it through for a while, it occurred to me that a fact that would nicely make sense of all their many bizarre actions would be if they were crack-heads.

Location: St Ann's, City of Nottingham, East Midlands, England, NG3 4QP, United Kingdom

Last View of Chase Farm

Posted by alexkemp on 14 March 2019 in English (English)

North towards Chase Farm

Above is one of the final of the set of photos taken May 8, 2017, within the edge of a stand of wild trees looking north-west towards the ruins of Chase Farm. It is close to a ridge of high ground which runs from Nottingham (to the South-West) towards the North-East, which is on the other side of the Farm, and which today is called Mapperley Plains. Only 150 years ago it was full of trees and was called “Sherwood Forest” (yes, that Sherwood Forest), was originally owned by the King and renowned for The Chase.

I’ve finally added all the stuff I surveyed back in 2017 to the map, and will very soon be free to begin mapping again after a gap of 2 years. The bypass whose development you can just see on the LHS is one of the things that needs mapping.

Hope you like the photo.

Location: Arnold and Carlton, Gedling, Nottinghamshire, East Midlands, England, United Kingdom

Mapperley’s Lime Tree Gardens Soakaway

Posted by alexkemp on 13 March 2019 in English (English)

Lime Tree Gardens Soakaway Lime Tree Gardens Soakaway (map)

This is the second time that I’ve come across a Flood Lagoon (the far-more-impressive first time was back on 20 June 2016 with the Carlton Foxhill Road Central Flood Prevention Lagoon. Thanks to lots of help provided on that First Contact, and after a phone call with Nav today (“my name is ‘Nav‘, like ‘SatNav’”) I’ve also got the inside skinny from Severn Trent on this one. And it is complicated…

There is a rectangle of grass (map) that encloses the whole thing:–

  • landuse = grass

3 sides have a metre-high fence (map):–

  • use menu: Highways|Barriers|Fence

The man-made structure that you can see in the centre of that photo above is two outlet pipes for a culvert that conducts surface water from the nearby estate’s roads into the soakaway:–

Lime Tree Gardens outlet

  • use menu: Man Made|Man Made|Pipeline
    (I was far less sure of using this one)

I’m much more confident of the mapping with the Soakaway basin (map) itself, which is a roundish depression within the centre of the rectangle:–

  • use menu: Water|Water|Basin
  • basin = infiltration

At the other corner opposite to the culvert outlets is a pumping station (map). This is the only item that Severn Trent actually had named (“Mapperley Lime Tree Gardens Pumping Station”):–

Mapperley Lime Tree Gardens Pumping Station

I’m unsure on this one, too. It is a rectangle of concrete surrounded by a 2m metal fence. That was easy to map. Then it has water-pumping equipment inside. Do I map that separate or together? In the end, I combined both sets of mapping. The works are designed to pump excess water out of the basin and into the nearby ditch.

  • use menu: Highway|Barriers|Fence
  • use menu: Man Made|Man Made|Water Works

The final items are a hedge + a ditch + an outfall pipe (map) within the ditch (using essentially the identical mapping for the culvert as above):–

Lime Tree Gardens pump outlet

Update

I’ve made some small changes following the comment from lakedistrict:– the pumpingstation has been changed from man_made=water works to man_made=pumping_station + substance=wastewater. That does not form part of the JOSM menu system, but does fit better to this situation and is documented within the wiki.

Location: Arnold and Carlton, Gedling, Nottinghamshire, East Midlands, England, United Kingdom

Re-learning JOSM :: More Advanced

Posted by alexkemp on 12 March 2019 in English (English)

BuildingsTools plugin (Wiki)
Terracer plugin (Wiki)

In the previous post I showed the basic steps of using the building_tools and terracer plugins together to map buildings with JOSM on to the OSM map, so make sure to read that if you get lost at all during this post. This section will give a pot-pouri of how to perform more advanced tasks using the same two plugins. It will start with 2 Bradstone Drive.

Drawing an L-Shaped Detached House

For some reason these are very common in England (this particular house is a 2-story modern house — built in the last 5 years — but bungalows in this style have been built like that since the ’20s). As a detached house, adding the L-shape can be done either before or after using terracer, but for all other houses any changes to the basic rectangle of a terrace must only be done after terracing the building.

  1. Draw the main frontage of the building, using (in this case) Esri World Imagery as a template
  2. Press ‘s’ and select the building just drawn
  3. Press ‘b’ and place the X-hairs above the common corner-node for both frontages and click
  4. As you now draw the side frontage, the mouse is shaping a box anchored on the common corner node.
  5. Shape the new box to the side-frontage, and click when complete
    (There are now 2 building rectangles positioned over all external borders of the house)
  6. Press Shift+j
    (“Join overlapping areas”)
  7. Remove the redundant nodes from 2 sides of the house
    (there are likely to be 2 nodes on top of each other on each side to be removed)

That’s it - an L-shaped house. A similar procedure can be performed for buildings with rear-extensions, etc.

Drawing Twin Garages

There is a bug in terracer from my point of view, in that it decides which angle of the rectangle of a building should be used to split the building (N-S or E-W, depending only on which is longer). This is crucial for twin-garages because terracer always chooses the wrong side, but it can also affect semi-detached houses and occasionally even 3-house terraces will split on the wrong axis. Here is how to fix them, using the twin-garages between 2 and 4 Bradstone Drive as the example.

Method 1 - rotate after splitting

This rarely works for garages; it is useful when the 2 buildings together are virtually a square.

  1. Draw the garage in the normal way
  2. Terrace in the normal way
  3. Press ‘s’ and select the building just drawn, if necessary
  4. Press Ctrl+Shift (the cursor changes to 2 x opposite-facing arrows)
  5. Whilst holding Ctrl+Shift down, click-and-hold on the screen and pull your mouse clockwise / anti-clockwise to turn the 2 garages through 90°

A word of warning
You need to make very sure that, during the procedure above, that only the house/terrace is selected. If, as one example, a 200 mile road is also selected (but off-screen whilst you work) then the road will also be twisted. That may not be what you want. Get into the habit of de-selecting everything before you select the one item that you want to move / rotate / whatever. JOSM is much more powerful than you may recognise, and will do exactly what you ask of it, quickly & easily.

Method 2 - draw & split at half-size then expand to full-size

This is a fudge until a “Use the other axis” checkbox gets added to terracer

  1. Draw the garage at half the length of the outline
    (so that the width is longer than the length)
  2. Terrace in the normal way → Press Ctrl+t for the terracer dialog
    → Ignore everything except…
    → (Segments = 2)
    → Building = garage
    → Press OK
  3. Pull down the bottom, central node between the 2 garages until you reach the centre of the bottom of the outline
    (if the garages have a roof-line then follow that line down)
  4. Pull down the bottom left- and right-nodes to the left- and right-hand sides of the bottom of the outline
    (do your best to keep both halves as rectangular as possible, but don’t freak out about it)
  5. Press ‘q’ (“Orthogonalise Shape”) (ie “make rectangular”)

That is it - twin garages.

Preventing a driveway from snapping to the garage

There is a driveway that runs from the road to the twin-garages in the previous section. A driveway connecting to a garage is not such a sin, but there are a ton of other occasions in which you absolutely do NOT want the object that you are drawing to connect to some other object(s).

  1. Press ‘a’ and hold down the Ctrl key
    (it is the second part that will stop the line that you are about to draw from sticking like lint to anything that it gets close to)
  2. Click the mouse button close to the garage, then draw it towards the road
  3. Release the Ctrl key
  4. At the road click the mouse button
    (That both finishes the line and attaches the node of the line to the road)
  5. Add keys to the line that you have just drawn:–
    → highway = service
    → service = driveway
    → surface = asphalt

Ensuring that a garage connects to the house

This is kind of the opposite to the previous section. The example is of the semi-detached houses 34 + 36 Bradstone Drive, which each have garages which attach to the side of the house on one side and run down the property line on the other. In the case of 34 Bradstone it is important that the garage does NOT snap to 32 Bradstone, whilst it must connect with 34 Bradstone. The ridge-line of each house + garage are useful markers to note where a particular garage starts & stops.

36 Bradstone is worse, in that it is a double garage; connected with 36 Bradstone on one side but NOT connected with 38 Bradstone on the other. If you are as anally-retentive as I am, then these features are important. After all, if you do not draw it right, how are the folks at 40 and 42 Bradstone going to be able to put their bins outside on collection day?

  1. Draw + terrace the semi-detached house in the ordinary way.
  2. Leave at least one of the semi-detached houses selected whilst drawing the garages
    (that causes the garage to share the same N-S / E-W axis as the houses)
  3. Hold down the Ctrl key whilst drawing close to 32 Bradstone so that the garage does not snap to the house.
  4. In contrast, you want the line of the garage to snap to the side of 34 Bradstone.
  5. When complete, there are at least 2 shared nodes between the garage & the side of 34 Bradstone; one near the centre of the house and one at the corner. These need to be bonded together, else the validator will complain of overlapping lines (‘lines’ are called ‘ways’ in JOSM).
  6. Select a node, then press ‘j’ (“Join Node to Way”)
  7. Repeat for the second node
  8. The garage on the side of 36 Bradstone is very similar with the added joy that it is a twin-garage and that needs to be sorted first; afterwards the two nodes need to be bonded to the house, same as before
  9. If ever a node bonds/snaps to a line and you need to unjoin it, then press Alt-j (“Disconnect Node from Way”)

Mapping Clones

Developers tend to use an identical architectural plan for multiple houses within an estate. If the local planning department has allowed the developer to build swarms of clone houses, that means that you are allowed to clone those houses when you map them. Do it like this (in this example as I map 20 to 30 Bradstone Drive):–

  1. Press ‘b’ and draw number 20 Bradstone
    (be very careful to get the orientation and measurements correct)
  2. Press ‘s’ and select the building
  3. Paste in the common tags (addr:postcode, addr:street, building + source)
  4. Click on an empty space then re-select the outline
    (this is to switch focus from the tags to the building outline)
  5. Press Ctrl+c (copy)
  6. Hover over 22 Bradstone and press Ctrl+v (paste)
  7. Repeat for 24, 26, 28 + 30 Bradstone
  8. Check that each house is nicely positioned
  9. Terrace each one to get the numbering, etc. correct
Location: Arnold and Carlton, Gedling, Nottinghamshire, East Midlands, England, United Kingdom

Re-learning JOSM :: Basics on using building_tools + terracer

Posted by alexkemp on 11 March 2019 in English (English)

BuildingsTools plugin (Wiki)
Terracer plugin (Wiki)

Nottingham City Council maintain an Adopted Highways Register which is fantastic for confirming both adopted & un-adopted streets local to me + house-numbers and, on occasion, housenames. The info comes from Ordnance Survey and therefore has a Crown Copyright. Because of that, it is important that I point out that I use the site to confirm my earlier survey results, rather than copying from it.

Adopted Highways register
A highway and a public right of way mean the same thing; they are both a “way” over which the public have a right to pass and repass at all times of the day and night. The term highway includes all carriageways (roads), streets, footways (pavements), footpaths, bridleways and byways. A highway may also include the adjacent verges.

This site shows which highways are adopted. If a highway is adopted it means it is maintained and repaired by Nottingham City Council. It also shows public rights of way (footpaths, bridleways and byways) both adopted and un-adopted which are recorded on the Definitive Map and Statement. This is the legal record of public rights of way in Nottingham at the time the Map and Statement were last updated (the “relevant date”).

You will therefore understand that the above is also the legal status that allows OSM surveyors in the UK to both survey and photograph from those highways anything that they can see.

Map Attitude

You are going to see me map all buildings with an area, rather than a node, and all the buildings are added to a street relation. The combination of the buildings_tools & terracer plugins makes both of these easy. I think that areas make the map look better, in that you can see how all the houses fit onto the street, but I do it like this mostly because JOSM makes it easy, and the map ends up looking much more professional.

terracer can be used for detached & semi-detached houses as well as terraces of 3 or more houses. The method used is identical for all 3 types. The main restriction is that only ‘normal’ (plain) rectangles can be terraced. Any rear-extensions, porticoes, turrets, what-have-you must be added after the building has been terraced.

As far as the associatedStreet relations are concerned, my (possibly biased) experience is that it makes searching very much more effective & quicker. It certainly ties some areas together in a satisfying manner. You can see what I mean by that by following these links for Denbury Court & Braunton Crescent.

Mapping Buildings Sequence

There are a large number of JOSM keyboard shortcuts and I would strongly urge you to learn them as to speed up your work (JOSM is a pain-in-the-arse otherwise). Here is step-by-step as I begin to draw a new set of buildings on Axmouth Drive, starting with 23 Axmouth Drive (half of a semi-detached house) which actually opens onto Braunton Crescent (the sort of thing that you find out only if you do the survey).

  1. Most of the time I need to remember just 4 keyboard shortcuts:–
    → s (to select something on the map)
    → a (to draw a node or a line or an irregular area)
    → b (to draw a rectangular building)
    → Shift+t (to terrace a building)
  2. Press ‘s’ then click on an empty part of the map to clear any selections
  3. Navigate to the part of the map to the point at which to draw the next house
    → + to zoom in
    → - to zoom out
    → Ctrl+<arrow-key> to move left / up / right / down
  4. Hold down Ctrl then select addr:postcode, addr:street + source (3 selections) within 30 Axmouth Drive then press Ctrl+c (or Ctrl+Insert) to copy
  5. Press ‘b’ then draw a box over the outline of the first semi-detached house on the corner of Axmouth Drive & Braunton Crescent
    → left-click on one corner of the house and pull the mouse down one side of that house
        (that action draws a red line as you move the mouse)
    → left-click on the opposite corner and you now have a line on screen
    → pull the mouse down to the opposite side and click again; now you have a rectangular box which has the tag for a building within it
  6. Press ‘s’ then hold down Ctrl (or Shift) and select the new building + one of the nodes at the north-end of the building + Axmouth Drive (3 items selected)
  7. Press ‘Shift+t’ to get the Terracer dialog
  8. Enter:
    Lowest Number = 21 (this is the end at which a node was selected)
    Highest Number = 23
    Interpolation = even/odd
    Segments (already = 2)
    Street Name (already = Axmouth Drive, due to the street being selected)
    Building = ‘semi’
    ☑ into ‘add to existing associatedStreet relation’ (even if it doesn’t yet exist)
    ☐ into ‘keep outline way’
  9. Press <Enter>
  10. (whilst both houses remain selected) press Ctrl+v (or Shift+Insert) to paste those earlier values into both houses

That’s it! On to the next building, then the next for several hundred more.

When a section is complete (I try to do one street at a time) press the uplift button (Ctrl+Shift+Up), add a suitable Changelog text and uplift to the server.

Easy. Just 30 million more buildings to do.

Location: Arnold and Carlton, Gedling, Nottinghamshire, East Midlands, England, United Kingdom

Re-learning JOSM :: Mapping Buildings :: Startup

Posted by alexkemp on 11 March 2019 in English (English)

One of the keys to using JOSM is to teach yourself to use the keyboard at the same time as you use the mouse. Once you have a small selection of keyboard shortcuts in your head things will go much quicker. kbd shortcuts

BuildingsTools plugin (Wiki)
Terracer plugin (Wiki)

2 mins tutorial: Drawing Buildings in JOSM
(from the HOT team; a little old, but still accurate)

Most of my time post-survey is spent mapping houses & roads - it is rare that a road is not already mapped, though naturally they always need checking and sometimes correcting. In contrast, it is rare in the areas that I survey that any houses are mapped at all.

Startup Sequence

Here is an account of my current startup sequence after launching JOSM from *menu | Education | JOSM (latest snapshot):–

  1. Items ready before JOSM launch:–
    → File directory with tracker contents
        (here ~/osmtracker/2017-05-28_10-53-09/)
    → GPX file (a GPS ‘breadcrumbs’ file of locations during the survey)
    → JPG files (GPS-located photos of street- & road-signs, housenames, etc.)
    → 3GPP files (GPS-located sound files)
    → .osm file (JOSM desktop saved from previous session)
  2. launch JOSM
  3. press alt-shift-D to switch continuousDownload OFF
  4. ctrl-O to load 2019-03-10.osm
    (desktop save from earlier session)
  5. press ctrl+U to update all data from the server
  6. File | Open Recent ⏵| 2017-05-28_10-53-09.gpx
    (the area covered on the screen is now huge, so press ‘+’ & ctrl+arrow-keys to navigate back to Axmouth Drive (last edit position))
  7. press alt-shift-D to switch continuousDownload back ON
  8. load menu: | Imagery | Esri World
    (it’s the best of a bad bunch)
  9. load menu: | Imagery | Postcode
    (needs to be last else it gets lost)
  10. load menu: | Windows | Mapillary image
    (it should already have a ☑, so press shift+comma)
Location: Arnold and Carlton, Gedling, Nottinghamshire, East Midlands, England, United Kingdom

Re-learning JOSM :: Installation

Posted by alexkemp on 10 March 2019 in English (English)

The Operating System (OS) is different on my computer now from when I last used JOSM, so it had to be installed from scratch. Here is a bunch of stuff that should be useful if you need to install JOSM:–

  1. JOSM == “Java OpenStreetMap Editor”
  2. JOSM Wiki
  3. Available for Windows, Linux, and macOS
    (you are strongly advised to use a desktop machine with a large display)
  4. Install instructions

The following will be install instructions for Devuan / Debian / Ubuntu:–

This is my OS:–

$ lsb_release -da    
Distributor ID:	Devuan
Description:	Devuan GNU/Linux 2.0 (ascii)
Release:	2.0
Codename:	ascii
  1. Linux install instructions
  2. JOSM requires a minimum of Java r8
    (ascii comes with r8 by default; I use openjdk java without any problems)

     $ update-alternatives --list java    
     /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java
    
  3. Create a new repository
    (use the following one-liner)

     echo deb https://josm.openstreetmap.de/apt alldist universe universe | sudo tee /etc/apt/sources.list.d/josm.list > /dev/null
     $
     $ cat /etc/apt/sources.list.d/josm.list
     # 'ascii' does not work in next line, hence 'alldist'
     deb https://josm.openstreetmap.de/apt alldist universe
    
  4. Install the JOSM key (another one-liner)

     $ wget -q https://josm.openstreetmap.de/josm-apt.key -O- | sudo apt-key add -
    
  5. Now update
    (either this or use synaptic)
    (be warned: JOSM gets updated almost every day)

     $ sudo apt-get update ; sudo apt-get install josm
    
  6. Launch JOSM via menu | Education
     
    (at first launch install / update the Plugins; use Global Preferences (f12) | Plugins)
    (fwiw these are the ones I have installed):–

    apache-commons    
    apache-http    
    buildings_tools    
    continuousDownload    
    DirectUpload    
    jna    
    Mapillary    
    terracer    
    waydownloader    
    
  7. Add UK postcodes via raggedred.net

    a. Go to Global Preferences (f12) | Imagery Preferences | Imagery Providers
    b. Add at the bottom:
    Menu Name: “Postcode (from raggedred.net)”    
    Imagery URL: “tms:http://www.raggedred.net/tiles/codepoint/{zoom}/{x}/{y}.png”
    c. Press OK
    

Re-Learning JOSM :: Intro

Posted by alexkemp on 10 March 2019 in English (English)

At this moment my profile shows 1,822 edits, all done with JOSM, so it would seem reasonable to assume that I know JOSM inside-out. Well, yes & no.

  1. The last time I went out surveying was 28 May 2017 (19 months ago)
  2. I never completed adding that survey into OSM
    (I got to Chedington Avenue on Spring Lane)
  3. After creating another batch of OSM Promotional Leaflets it remains only for me to finish the necessary editing on the last survey. Then I can get on the road again for some more surveying.
  4. I fell down dead 20 years ago so I try not to be too hard on myself if I forget things
    (23 February 1999 — heart went into fibrillation — on awakening I did not know where I lived nor worked, and had the memory of a goldfish (’bout 4 minutes))

My OS is different than the last time I used JOSM, so everything needed to be done from the beginning, again. My thought was to make use of this opportunity to give advice to those starting from scratch and trying to learn how on earth to use this programme. Uniquely, I’m both well-experienced and a complete novice in using it at the same time, and that may help give me an insight into what newbies may need to know.

This will be just an introductory post. The meat of the matter will start at the next post.

I do not intend to structure the help too strongly. It will be a bunch of disparate stuff as I come across it whilst editing. I strongly urge others to add in their own useful titbits in the comments, or to correct wrongful info.

Location: Thorneywood, Sneinton, City of Nottingham, East Midlands, England, NG3 2PB, United Kingdom

A new record for Mapillary - 62 blurs on a street sign

Posted by alexkemp on 18 February 2019 in English (English)
  1. Mapillary seems to be Losing It’s Marbles
  2. Git Along, Little Dogies
    (speeding up renaming files)
  3. I’m losing my Marbles because of Mapillary
  4. A new record for Mapillary - 62 blurs on a street sign

a new record

I know that on first sight that it does not look like it, but this street sign (above) at the corner of Foxhill Road & Forester Grove near Carlton, Nottingham contains 62 blurs! There are so many that the listing (see the window in an earlier diary) stops at #44 with all the rest on the LHS out of site below the window.

A piece of advice from an old man:
Make sure that you admit your mistakes, and try to do so promptly. Otherwise you will find yourself unable to change and making the same stupid errors over & over again, which is most boring.

Location: Arnold and Carlton, Gedling, Nottinghamshire, East Midlands, England, United Kingdom

I'm losing my Marbles because of Mapillary

Posted by alexkemp on 17 February 2019 in English (English)
  1. Mapillary seems to be Losing It’s Marbles
  2. Git Along, Little Dogies
    (speeding up renaming files)
  3. I’m losing my Marbles because of Mapillary
  4. A new record for Mapillary - 62 blurs on a street sign

The picture below was shot on 15 July 2016 on the corner of Carnarvon Grove & Cavendish Road within the parish of Carlton, Nottingham. Like so many other Nottingham street signs it has been vandalised, in this case quite recently, and the vandalism was performed by Mapillary.

I suspect that Mapillary may have jumped the shark. They are certainly losing my affections. After about 10 12-hour days spent editing a few thousand blurs within 69 sets of photographs, yet without any action on their part, I recently sent them an email:

From: Me
Subject: Re: [Mapillary] Re: Re: [Mapillary] Blur editor ineffective
To: Mapillary <support@mapillary.com>

More problems with the Blur Editor window:

The left-hand window (it contain the “Add new blur” button) fills up when > 44 blurs are within the window. It needs a scroll-bar to maintain access to all components.

Thank you.

That street-sign above has a single blur within the centre of the sign which covers the entirety of the sign. It then has a further 28 blurs inside the first blur. That is so imbecilic that I keep having to lift up my jaw to close my mouth, but the most staggering fact is that the combination of all 29 blurs fails to blur the sign. Further, their own guidelines say not to blur “Street, traffic and information signs” How dumb is that?

I’ve lodged 6,200 photos with Mapillary, all shot whilst walking the streets of Nottingham suburbs & villages. They recently set a new algorithm in motion which vandalised 32% of those photos with unnecessary blurs (my evidence is that 92% of the new blurs are unnecessary).

I’m about halfway through trying to edit the blurs out of my photographs. I started with the most recent ones and have now just reached the photo above. No response yet from Mapillary after ~10 days as to whether they accept that the new blurs are a mistake, nor any action on my blur edits.

Location: Arnold and Carlton, Gedling, Nottinghamshire, East Midlands, England, United Kingdom

Git Along, Little Dogies

Posted by alexkemp on 15 February 2019 in English (English)
  1. Mapillary seems to be Losing It’s Marbles
  2. Git Along, Little Dogies
    (speeding up renaming files)
  3. I’m losing my Marbles because of Mapillary
  4. A new record for Mapillary - 62 blurs on a street sign

Git Along, Little Dogies (Be warned that today’s Diary is most whimsical)
(PS That picture above is linked to a “Family singalong”)

There is a house on Kenrick Road, Porchester Gardens, Nottingham that is called “Fool’s Jig”, and there is a link in the map to a picture lodged within Mapillary of that House-Name (below).

Fool’s Jig

It is worth saying that that seems to be the only house in the world called “Fool’s Jig”. The name possibly refers to Will Kemp, one of the original Kings’ Men (the troupe that performed Shakespeare’s plays), the first to play Lear, and who was most famous for his nine-day jig from London to Norwich plus dying in poverty & neglect. There are also a dozen different interesting things in earlier Diaries about locations close to this house, including the way that the footpath Donkey Steps actually continues as footpaths across nearby houses and then used to run close to a neighbouring house and finally across to Porchester Road, but was stolen by that householder. However, none of that is the subject of this Diary.

How to Quickly arrange JPEGs within Directories

This is the command using exiftool to do the business:

exiftool '-FileName<CreateDate' -d '%Y-%m-%d_%H-%M-%S.jpg' .

The one-liner above will rewrite the names of all files within the directory (in this case it assumes that they are all JPEGs). An example resultant filename for the house-name above is 2016-09-28_10-46-07.jpg. It is also possible to rename all files and move them into directories, creating those directories as required (but not with this specific command).

EXIF Dates in Excruciating Detail

The exiftool uses xmp names in some instances, whilst the identify command-line tool from ImageMagick uses the standard EXIF field-names throughout (see exif r2.3 (PDF) (simple) or exif r2.3 (PDF) (full)):

$ identify -verbose 2016-09-28_10-46-07.jpg | fgrep -i datetime
    exif:DateTime: 2016:09:28 10:46:07
    exif:DateTimeDigitized: 2016:09:28 10:46:07
    exif:DateTimeOriginal: 2016:09:29 13:03:25

$ exiftool -a -G1 -s -time:all 2016-09-28_10-46-07.jpg    
[System]  FileModifyDate      : 2019:02:13 21:47:09+00:00    
[System]  FileAccessDate      : 2019:02:13 21:47:09+00:00    
[System]  FileInodeChangeDate : 2019:02:15 01:02:24+00:00    
[IFD0]    ModifyDate          : 2016:09:28 10:46:07    
[ExifIFD] DateTimeOriginal    : 2016:09:29 13:03:25    
[ExifIFD] CreateDate          : 2016:09:28 10:46:07

There are 3 date-times available for every OS:

  1. cTime (metadata modification time)
    p9: DateTimeOriginal (EXIF IFD) = Date and time when the original data was generated
    FileInodeChangeDate - created by system & altered by system
    (same for identify + exiftool)
  2. aTime (system access time)
    p9: DateTimeDigitized (amp:CreateDate) = The date and time when the image was stored as digital data
    FileAccessDate - certain system accesses will update aTime but leave mTime unchanged
    (different for identify + exiftool)
  3. mTime (data modification time)
    p6: DateTime (xmp:ModifyDate) = Date and time when the file was last modified
    FileModifyDate - when file-data was last updated
    (different for identify + exiftool)

So what’s the point?

There are two things in this:

  1. My Computer crashed & I lost some data due to a bad backup
  2. The original mobile went weird on some shots

Mapillary offers a “Download unprocessed original” button for all photos that a user has uploaded. There is no Download all button, so it is one-by-one. Getting the filenames right was sending me up the wall. Then I was able to work out the command above & that was sorted.

Now look closely at the filetimes above and you will see that the DateTimeOriginal is different to all the others (and wrong). It was the same for a whole bunch of photos (iirc those were all shot with a mobile that eventually proved to be faulty). The bad metadata was causing problems with Mapillary, as it uses that field to tie all photos from a single user together. Whoops. I made sure to avoid that by using CreateDate myself.

Location: Porchester Gardens, Arnold and Carlton, Gedling, Nottinghamshire, East Midlands, England, United Kingdom

GPS to have it's 2ⁿᵈ Y2K Moment on April 6 This Year

Posted by alexkemp on 12 February 2019 in English (English)

Is Location on my Smartphone going to die? OS updates may well be available, but Vodaphone stopped providing them 0 secs after I bought it.

GPS Rollover This April 6

(boring detail from the Register link above follows)
GPS satellites contain an atomic clock, and the signal that they put out contains a timestamp derived from that clock. The timestamp is an inherent part of the way in which a device Location is calculated from the accumulated GPS signals. However, that Timestamp stores the week number using ten binary bits…

Ten binary bits == 2¹⁰ == 1,024 weeks ≈ 20 years

The first GPS satellite launch was 1978. The first epoch was 6 January 1980 & the first rollover was midnight UTC Sun 22 Aug 1999 & the next will be the first Saturday in April 2019. Yikes!

Extra

If you want to see the timestamp for a JPEG with embedded Location info, then do this (and note: this was done because the time went weird for this set of photos (all 3 timestamps should be the same)):–

$ identify -verbose 2016-10-08_11-27-27.jpg | fgrep 'exif:DateTime'    
exif:DateTime: 2016:10:08 10:27:27
exif:DateTimeDigitized: 2016:10:08 10:27:27
exif:DateTimeOriginal: 2016:10:08 21:58:52

Phew! We are all Saved!

I’m indebted to TheSwavu (see Comments) for a link to wired.com detailing how everything went fine (?) on the last time this happened in 1999:–

At 8 p.m. EDT Saturday (21 Aug 1999), the clock was reset to zero

The US Coast Guard said it was unaware of any serious distress calls from boaters related to malfunctioning GPS receivers.

He said fewer than 12 Coast Guard cutters, aircraft, boats, cars, and other auxiliary vessels reported a glitch, however fleeting, when their GPS receivers failed to update automatically. Fixing those short-lived glitches typically required nothing more than powering down a GPS receiver to get re-synchronized with the satellites, McPherson said.

In Japan, Pioneer Electronic, one of several car navigation system makers, said it had received several hundred phone calls Sunday from customers whose devices failed because they baulked at the clock resetting. Some of the older models, made before 1995, were now showing the date of 7 January, 1980, as if this were Day 2 back in the original week 0. But they were otherwise showing the correct position and time, he said.

So there we are. Only 12 US Coast-Guard vessels & several hundred Japanese customers. So it is all going to be OK. As long as you do not work for the Coast Guard. Or are Japanese (or, god forbid, are a Japanese Coast Guard). And if you do suffer a glitch, then just turn it off then back on again.

Location: Thorneywood, Sneinton, City of Nottingham, East Midlands, England, NG3 2PB, United Kingdom

Mapillary seems to be Losing It’s Marbles

Posted by alexkemp on 9 February 2019 in English (English)
  1. Mapillary seems to be Losing It’s Marbles
  2. Git Along, Little Dogies
    (speeding up renaming files)
  3. I’m losing my Marbles because of Mapillary
  4. A new record for Mapillary - 62 blurs on a street sign

Mapillary is a location-photo-sharing site dedicated to inter-working with OSM & I like it a lot.

Mapillary is the street-level imagery platform that uses computer vision to fix the world’s maps.
450.7 million images; 6.5 million kilometers covered

I am registered on Mapillary since Apr 10, 2016 and it tells me that in that time I have uploaded 6,200 photographs shot whilst walking across 244.7 km of Nottingham’s suburbs. Most of those photos are registered as links with various PoI on the map, which means that someone browsing OSM can find a link attached to a Nottingham point-of-interest that will instantly let them see a blurred photo of what it looks like 32% of the time.

Blurred Photos

At the moment that I am writing these words the following photo has 33 auto-blurs attached to it, 32 of which affect the street-sign on the right (the 33rd is on the brickwork to the right):–

blurry street-sign

At this moment I am going through each & every one of those 6200 photos removing the stupid-blurs. Nothing will happen until at least next Monday (11 Feb), as all the blur-removals are actually blur-removal requests (a Mapillary admin has to agree/refuse the request).

33 blurs is bad, but not the worst - one photo had more than 40 blurs (a patch of shrubbery) (whilst another just now had 45 all on top of another street sign) (and the record worst has 62), all of which were stupid-blurs.

So what is going on?

Mapillary has always edited uploaded images before showing them:–

What Mapillary blurs:

  • Faces
  • Vehicle license plates
  • Inappropriate or private parts of the image

What Mapillary does not blur:

  • Whole people
  • People from behind when the face is not seen
  • Billboards, advertisements, brand labels
  • Street, traffic and information signs

So what is the problem?

I have zero problem with Mapillary enforcing privacy blurs. I’ve dealt with too many MPGs spontaneously combusting just because I’m stood outside their house asking if I may take a photo of their house-name not to know that the English have a particular issue with Google StreetView (if you get my drift through all those disparate topics).

In the early days Mapillary kept blurring out street-signs, and it has taken quite a few years for them to stop doing that. Most of the time now it seems to work OK. However, the Blur Algorithm appears to have recently been put onto steroids [update: actually Apr 2018; see bottom], in that it is blurring areas so small that they can barely be seen. Worse, the programming has schoolboy-level errors; as just one example, some of the street-sign blurs are quite large yet inside that area may be yet another 5 smaller blurs - what is the point of that?

Look at the photo below; it is the example used in the help-page to illustrate what the privacy-blur edit-page looks like. Those orange/green/white numbered circles locate the presence of a blur enabled by the Mapillary privacy-bot.

Mapillary privacy-blur edit-page

Each blur has to be removed one-by-one using a mouse, no keyboard allowed. This is tedium ratched-up to mind-numbing proportions.

  1. Hover over a blur
  2. Move the mouse up to a combo of green tick✔ + orange cross✘ that then appears
  3. Hover over & click on the cross
    (maybe you are hovering over the cross, maybe not)
  4. Next blur…

My constant experience across the last two days has included a bunch of 5-or-more of those circles located on top of each other in such a fashion that I cannot click on the ones underneath. Or of circles located under image navigation features (arrows & chevrons, forward/back arrows or attribution-feature); those are all located in top-of-z-order, which means that nothing underneath them can be clicked on. These are what I am calling “stupid-blurs”.

The Modified Date for the Blur Help Page is January 09, 2019 13:01 (there is a set of blurs on all earlier photos in addition to these recent blurs; those earlier blurs are no longer editable). I therefore assume that the blur-bot was very recently unleashed on all 451m Mapillary photos to enforce further blurs. Or, more accurately, further stupid-blurs. Get more intelligent more quickly, please, Mapillary-bot!

Stats

Being a scientist I thought some measure of the problems was required.

I’m working my way from latest to earliest and have already gone through 26 sets of photos. Here are the accumulated stats from the 27ᵗʰ set of 111 photos, shot on 31 January 2017 in the streets near Phoenix Avenue close to Gedling:–

  • Total photos: 111
  • Contains blurs: 35 (32%)
    (following applies to blurred photos only)
  • smallest # blurs: 1
  • largest # blurs: 25
  • Total blurs: 233 (avg: 7/photo)
  • Valid blurs: 19 (8%)
  • Stupid blurs: 214 (92%)

Addendum

Tues 12 Feb: Mapillary have sent me an email to say that the team is looking at it.
Tues 19 Feb: Map email: “The dev that would be the best to investigate and answer these questions is currently on vacation and will be back after this week.”
Fri 22 Feb: Shortly after 11pm I corrected the last blur on the last set of photographs; earlier in the evening I had saved the last ‘lost’ photo to my home hdd. Mapillary have still to apologise for the many bugs & errors, or acknowledge any of these blur corrections, let alone act on them. They say in a recent email “our algorithms work on street level imagery at best and tend to fail with close-up imagery, such as close-up images of signs.” Ah, spit.
Sun 24 Feb: The stupid-blurs appear to have started on 19 Apr 2018 and took 3 weeks to vandalise all images (500 images/second).

The goal is to blur all identifiable faces and license plates. At the same time, we need to preserve important information in the images for mapping. This makes it a challenging problem.

The resulting privacy blurring model is able to detect 99.9% of all identifiable faces and license plates while wrongly blurring only 0.078% of the pixels. This means that on average, we miss just one out of 1,000 identifiable faces and license plates.

I checked carefully, and the words “Collateral Damage” and/or “Bugs” are not yet used at any point.

Location: Arnold and Carlton, Gedling, Nottinghamshire, East Midlands, England, United Kingdom

Adding a Preview JPG + crop-marks for printing

Posted by alexkemp on 5 February 2019 in English (English)

This is a another small update to a sequence of posts on updating some small leaflets originally produced by Andy Allan:–

  1. Fonts missing from OSM Promotional Leaflets
  2. Github HowTo: Clone Repository then make a Pull Request
  3. Printing OSM Promotional Leaflets
  4. Update to OSM Promotional Leaflets due to Padding errors
  5. Adding a Preview JPG + crop-marks for printing

OSM leaflet preview

This is how to add crop-marks and bleed-marks to a production-PDF for printing (the PDFs within GitHub do NOT currently contain any crop-marks and, whilst they contain 3mm bleed, they do NOT contain bleed marks either):–

  1. Load your production-ready SVG into Inkscape
    (see earlier pages for links if you need to install)
  2. Under r15371 (the most recent) go Extensions | Render | Layout | Printing Marks…
  3. Put ticks in Marks | Crop Marks + Marks | Bleed Marks
  4. On Positioning tab set:

      Set crop marks = `Canvas`
      Set       unit = `mm`
      Set     offset = `0.000`
      Bleed Margin
              Top    = `3.000`
              Bottom = `3.000`
              Left   = `3.000`
              Right  = `3.000`    
    
  5. Click ✔Apply (and wait a little while it works)

You will see that both options in the JPEG above contain both sets of marks; that is really quick & easy to produce.

This was a temporary edit as I was simply producing two PNGs for a preview image of the 2 options for the leaflet front (the printers had asked for 3mm bleed but NOT for crop-marks). As a final visual guide I added 3 x vertical dotted-lines to show where the leaftet is folded. The next step is to export the finished product as a PNG file.

Exporting is easy, but the range of options is a little terrifying. Fortunately the defaults worked:

  1. Select File | Export PNG Image…
    (a dialog appears)
  2. Select Drawing as Export area (after checking that dimensions are correct under Page)

After setting a name (I left the default of name.svg.png) all the other defaults proved to be correct. That was NOT a coincidence. I had previously taken every precaution to make the page-size accurate, then apply (internal) 4mm padding + make (external) 3mm margin available (which is the bleed, set when the PDF is produced, although more than that actually exists). Thus, the defaults were all setup accurately.

Finally, push ✔Export and check the result (there is zero confirmation from Inkscape).

I used GIMP to put the two PNGs together and exported as a JPEG. The GIMP master file has been uploaded to try to make it easy for anyone that wants to make changes. The whole thing has been pushed to GitHub as a further commit. I hope that some will find it useful.

Good luck!

PS
This looks like a good link if considering PDFs & crop-marks (the summary is: “PDFs are best for printers, use crop-marks but no others, use 300dpi for raster items”). It also said “Acrobat Reader is the best application to use when opening and viewing a PDF file.”. I almost spat tea all over my monitor when I read that. It points out that printers will probably get in a huff if you include colour registrations, etc. (I thought that they looked splendid, but would not be included in the PDF in any case since they are located outside of the bleed).

Modern SVGs use 96dpi. The ones used by Inkscape r9886 (the latest version available when Andy first designed these leaflets) used 90 dpi. That change from 90dpi to 96dpi caused hours & hours of re-formatting when I updated the whole GitHub package from r9886 to the current r15371.

It makes sense when printing to go down in resolution if needed for some reason, but you cannot go up in resolution and then see any actual difference in the end product. That means that, if the PDF is going to contain any raster images (and PNG are raster images), then the SVG source files will need to reference 300dpi images to get a 300dpi end product.

To try to get a measure of the problem:

$ identify globe_part_halo.png london.png    
globe_part_halo.png PNG 3208x3880 3208x3880+0+0 8-bit sRGB 8.236MB 0.000u 0:00.000
london.png PNG 2479x1356 2479x1356+0+0 8-bit sRGB 2.119MB 0.000u 0:00.000

pixelto conversions at 96dpi + 300dpi
(leaflet pages are 73mm wide x 105mm high):

pixels    96dpi  300dpi
------  -------  ------
 3,880  1,027mm   329mm
 2,479    656mm   210mm

It looks like it should be possible to use the current source files to print at 300dpi but I am unsure of the methodology to get from the current SVG dpi to PDF dpi. Advice appreciated.

Location: Thorneywood, Sneinton, City of Nottingham, East Midlands, England, NG3 2PB, United Kingdom

Update to OSM Promotional Leaflets due to Padding errors

Posted by alexkemp on 31 January 2019 in English (English)

This is a small update to a sequence of posts on updating some small leaflets originally produced by Andy Allan:–

  1. Fonts missing from OSM Promotional Leaflets
  2. Github HowTo: Clone Repository then make a Pull Request
  3. Printing OSM Promotional Leaflets
  4. Update to OSM Promotional Leaflets due to Padding errors
  5. Adding a Preview JPG + crop-marks for printing

OSM flyers

Diary website 500 errors

Yesterday & earlier today this website was producing a HTTP 500 errors (server error, explicitly reported as an error in communicating with a 3rd-party server). It occurred as I attempted to update my post on Printing OSM Promotional Leaflets. I sent an email to the Server folks.

A few minutes ago I succeeded in updating that post but again noticed occasional problems, which appeared to be connected with the Markdown used on this site. Openstreetmap.org is also sometimes very slow in loading a page.

Those paragraphs above were a quick heads-up for others that may be attempting to post on this site. Now back to the OSM Promotional Leaflets…

Update to OSM Promotional Leaflets due to Padding errors

Yesterday I should have reached the end of this little series of posts as the updated leaflets were printed & successfully delivered to my next-door-but-one neighbour. However, looking at them closely I realised that the padding was bad in places. In other words, a bit of a Curate’s Egg.
good in parts Right Reverend Host: “I’m afraid you’ve got a bad Egg, Mr Jones!”;
The Curate: “Oh no, my Lord, I assure you! Parts of it are excellent!”

GitHub Changes

I spent a few hours making a comprehensive fix, then uploaded those changes to GitHub. I’ve changed the GitHub layout to try to make it easy to spot the original (now out-of-date) SVG files within a Inkscape-r9886 directory, whilst the updated files are within a Inkscape-r15371 directory. The pre-prepared PDF file for printing (‘osmflyer.pdf’, a combo-pdf of both top & bottom parts of the leaflet) is within the top-level directory.

Within the Inkscape-r15371 directory, I have completely re-done the Guides for each SVG (by default, visibility for Guides is switched OFF so, until I looked, I did not know that Andy had included any). Each Guide now contains an explanatory label. They are:–

  1. 4 x Guides at page-edges (Page top, bottom, left & right)
  2. 4 x Guides at page-edge-padding (4mm Padding top, bottom, left & right)
  3. 3 x Guides at each fold (Fold centre, left & right
  4. 6 x Guides at each fold-padding (4mm Padding at left-fold-left & right, centre-fold-left & right + right-fold-left & right

Complicated to explain in words, easy to understand when viewed inside Inkscape.

Once I had got all the Guides in place it was relatively easy (if immensely fiddly, and especially on the reverse side) to re-position and/or shrink content within the Guides rather than spilling over towards the page-edges.

Update Feb 1

Having just added info on the SVG Guides (above), and having loaded Inkscape-r15371/osmflyer1.svg to refresh my mind, I could not stop myself making some little improvements to the text layout between the Guides & re-uploading all changes.

If/when I use up all the current leaflets & get some more then I shall fix the already-out-of-date photos.

Location: Thorneywood, Sneinton, City of Nottingham, East Midlands, England, NG3 2PB, United Kingdom

Printing OSM Promotional Leaflets

Posted by alexkemp on 24 January 2019 in English (English)

This is the final in a sequence of posts on acquiring some small leaflets originally produced by Andy Allan:–

  1. Fonts missing from OSM Promotional Leaflets
  2. Github HowTo: Clone Repository then make a Pull Request
  3. Printing OSM Promotional Leaflets
  4. Update to OSM Promotional Leaflets due to Padding errors
  5. Adding a Preview JPG + crop-marks for printing

OSM flyers

Five years ago Andy Allan produced a GitHub repository of a design for a credit-card sized (A7: 105mm high x 74mm wide) 8-page leaflet based on a Frederik Ramm German design, but updated for the UK (see above + at openstreetmap-promotional-leaflets). Andy had very kindly sent me a bunch of those leaflets on May 2016, and told me that they cost GBP £95 for 5,000 leaflets direct from the printers or GBP ~£165 for 10,000.

There is always an occasion whilst mapping when someone comes up & says “What do you think you are doing?”. I found that those leaflets helped to cut all the explanation to a minimum, and removed almost all aggravation. After many months & thousands of leaflets handed out I was down to the last few.

Near the end of last year I went to a local printer to get some more & discovered that they did not have the leaflet fonts on their Mac, so could not print from it. At home, the whole collection turned out to need updating (no surprise after 5 years) (cue 1st diary post).

Andy suggested to make a pull request. I said “That’s a great idea” then went home to try to work out what he was talking about (cue 2nd diary post).

The story now reaches last Monday 21 January. I had updated the original GitHub collection for the latest version of Inkscape + sourced all the fonts + fixed the SVGs + produced PDFs suitable for printing. Now for the proof of the pudding:– will it print OK?

In the end I got 6 quotes:–

  1. print24.com/uk 5-years ago by Andy Allan
     5,000 — £95
    10,000 — £165
  2. John E Wright
     5,000 — (still waiting)
    10,000 — (still waiting)
    25,000 — (still waiting)
  3. Portland Printers, Nottingham University
     1,000 — £245
     5,000 — £327
    10,000 — £376
  4. Fineprint, Nottingham
     5,000 — £206
    10,000 — £291
    25,000 — £532
  5. Big Thinkers, East Bridgford
     5,000 — £154
    10,000 — £262
    25,000 — £564
  6. print24.com/uk
     5,000 — £58.11 + £10.33 delivery
    10,000 — £103.89 + £19.85 delivery
    25,000 — £241.21 + £42.20 delivery

The fact that print24 will print 25,000 leaflets for less than Portland will charge for 1,000 of the same leaflets still blows my mind. John E Wright was the first printer that I approached, but they are waiting for a 3rd party to quote on folding the leaflets and cannot give a price until then.

I found John E Wright & Big Thinkers to be the most professional of all the folks I approached. Those on support at print24 were the least professional & perfectly ignorant about print. They appeared to be natives in India speaking Inglish, badly.† The managing director was Ali Bazooband, and the printing press was somewhere in Germany. If Andy had not already made use of them I would never have considered them, and worried constantly that it may be a scam. We shall find out when/if the parcels arrive, for I gave them (unitedprint.com UK Ltd. aka print24) the order for 5,000 leaflets.

Some advice about print24:

  • I was advised to create an account on the website (print24.com/uk) by a gentleman at the support number but did not - you may wish to avoid my mistake.
  • I was able to get to the point on Monday 2019-01-21 16:18 of being posted an email confirming the order, but whilst the PDF that I uploaded at order creation both registered & then showed on-screen, print24 never received it.
  • The emailed order contained IBAN, Swift, Bank code & Account number for their bank. My bank (NatWest) told me there was a £20+ charge for a £68.44 IBAN/Swift transfer. I made a charge-free Bank-code/Account-number bank-to-bank transfer on Tuesday 2019-01-22 11:02 & emailed print24 the time of transfer. They never replied.
  • They sent me a “Missing payment for order number” email on Wednesday 2019-01-23 01:02. I scanned my Payment Confirmation and emailed it to them.
  • The same email said “In addition the printing files are missing” and advised me to email the PDF to a specific email address. I did so 4 times with different subject-lines and each time received a reply saying “Customer and order number do not match or are incomplete” and that the email had been discarded. I emailed support for a fix & got no reply.
  • I phoned the support line again. It said “Welcome to our 24/7 support line. Unfortunately the office is closed at this time”. I was getting rather twitchy by this point.
  • At 10:24 Wednesday I got an email saying “We have received the payment but unfortunately not your artwork. Could you please upload it again with the data upload on our website?”. I did what they asked and, presumably because it had now registered my payment, it now worked with a confirmation…
  • At 12:49 the email said “Confirmation of file received osmflyer.pdf”
  • At 13:06 the email said “the print data for your order was checked as part of the basic data check – the countdown clock is running! Delivery by 31/01/2019.”
  • Mon 28 Jan 19:04 UTC print24 sent an invoice for the order
  • Mon 28 Jan 19:26 UTC print24 sent a Shipment Confirmation order together with a link to ‘tracking.dpd.de’ to allow the shipment progress to be followed. This says:–

      Date       Time   DPD parcel centre    Parcel status
      28.01.2019 14:53  Kesselsdorf (DE) ... In transit.
      28.01.2019 13:50  DPD data centre      Order information has been transmitted to DPD.    
    
  • Tue 29 Jan 03:03 UTC the email subject was: print24-guest access expires in 24 hours; then: “You have not placed an order through your print24 guest access for a long time. Take the chance to place additional orders now with your account. If you do not want to order additional products with your guest account, we will delete your data immediately.” That is worrying.
  • Tue 29 Jan 22:59 UTC the email subject from dpd.co.uk was: due for delivery on Wednesday 30th January; then: “On the morning of the Wednesday 30th January you’ll receive a 1 hour delivery time slot and you’ll be able to live track your order on its way to you.” (I was warned through reading comments on the website that they tend to deliver 1 day early). No notice yet how many parcels in my delivery.
  • Wed 30 Jan 06:02 UTC Observing the conjunction of Venus & Jupiter pre-dawn, low in the south-east sky, with the Waning Crescent of the Moon not far away above & to the right. The skies are perfectly clear, which is why my car is covered in frost from front to rear.
  • Wed 30 Jan 07:14 UTC Text from DPD: “Your order will be delivered 30 Jan by DPD. We’ll send your 1 hour time slot by 11am. Not going to be in? (a dpd.co.uk url)”
  • Wed 30 Jan 09:45 UTC Text + email from DPD: “DPD driver Penko will deliver your dpd order today 14:41-15:41. Not going to be in? (same dpd.co.uk url)” ‘tracking.dpd.de’ now says:–

       Date       Time   DPD parcel centre    Parcel status
       30.01.2019 14:57  Nottingham (GB)      Delivered.
                                              Evidence >
       30.01.2019 09:32  Nottingham (GB)      Out for delivery.
       30.01.2019 01:35  Nottingham (GB)      At parcel delivery centre.
       29.01.2019 19:15  Hinckley, Leices ... In transit.
       28.01.2019 14:53  Kesselsdorf (DE) ... In transit.
       28.01.2019 13:50  DPD data centre      Order information has been transmitted to DPD.   
    
  • Wed 30 Jan 14:59 UTC Text + email from DPD: “Your dpd parcel has been delivered today by DPD, to your neighbour (his name) at (house-number).” I was sat 2 feet from the bell-push loudspeaker all afternoon. Other people can use it. How come all delivery folks cannot?
  • The print quality + paper + numbers seem good. I can now tell, however, that I made some minor mistakes with Padding on both sides when I updated the SVGs. Others may not notice, but I certainly do. I’ve fixed that & made further commits to the Repository.

I’m basically happy with print24. They are a long way from perfect, but have produced what they promised, when they promised it, at an exceptionally low price (available Europe-wide). (Thu 31 Jan): I’ve added replacements for Andy’s photos to show what these updated leaflets look like. They are not as good as Andy’s photos, but do show more accurately what the leaflets look like as available today.

†Native English can have difficulty understanding dialects of those from other districts of England. That happened to me 50 years ago in Northumberland when the car got lost after a diversion due to road-works (converting the A1 Great North Road to the A1(M)); no-one in the car could understand the reply from the local whom we asked for directions, but thanked him politely & followed his gesticulations. That is why many English folks speak two languages: their local lingo & ‘polite’ English (for visitors from the next-door town/district).

Github HowTo: Clone Repository then make a Pull Request

Posted by alexkemp on 12 January 2019 in English (English)

Most things are easy once you know all the jargon and have done it for a while. That broke down for me with Github. It seemed that everybody just assumed that everybody else already knew all the jargon, and nobody bothered to explain how to do anything step-by-step.

  1. Fonts missing from OSM Promotional Leaflets
  2. Github HowTo: Clone Repository then make a Pull Request
  3. Printing OSM Promotional Leaflets
  4. Update to OSM Promotional Leaflets due to Padding errors
  5. Adding a Preview JPG + crop-marks for printing

I cannot explain everything about git nor Github, but I’ve just successfully created my first git Repository on Github by forking another repository, have cloned it to my local Desktop, made three commits & also a pull request. So, whilst everything is still reasonably fresh in my mind & I have the history available to report it all to you, here is some of what all that shit means and how to get into it if you want to do something similar.

Brief Background & Overview

Near the end of last year (2018) I tried to use Andy Allan’s OSM Promotional Leaflets to get some little giveaway-leaflets printed, and failed. It was missing some crucial items & also needed updating, so I tried, and eventually succeeded in updating it, and also made a post about that: Fonts missing from OSM Promotional Leaflets.

I’ve found Andy to be tremendously helpful in the past, so I was also privately in conversation with him about his Repository. He suggested that I make a Pull Request as to add links for the necessary fonts to the README file. I thought that that might be a great idea, just as soon as I could find out what a Pull Request was, and then how to do it.

Git + Github Overview

My home computer is powered by Devuan (a variant of Debian), and I have been working with various versions of Linux for the last 20 years, ever since I realised that the corporate world had decided that I was too old at 50 to be viable as an employee so decided, with essentially zero prior experience, to become a network engineer in charge of an enterprise, internet-connected computer. The enterprise + the computer were entirely owned by me, but that was fun! In part.

Linux is one practical realisation of the power of the Open Source movement (as, in another way, is OSM, and the Scientific community). One fundamental feature of Open Source software is that it is freely available to all to Fork (which means, to copy in it’s entirety and to then develop it further in the way that you see fit). Of course, a better way may be to feed bugfixes/improvements back to those developing a software suite so that it can continue to improve, but perhaps you get the point. git & GitHub is a means to implement that process.

git

a version control system for tracking changes in computer files and coordinating work on those files among multiple people

It was originally written by Linus Torvalds (the same guy that wrote Linux, and who still keeps an iron grip over the Linux kernel).

GitHub

a web-based hosting service for version control using Git

Founded in 2008 as a USA LLC, and sold 10 years later to Micro$oft for the silly sum of USD $7.5 billion. In June 2018 it had 28 million public repositories. That figure went from 46,000 one year after being founded (Feb 2009) to 1 million (Jul 2010) & 2 million (Apr 2011).

It is free to create a username on GitHub. It is also free of financial charge to create a Public Repository, but a Private Repository requires a monthly payment (‘repositories’ are places to store files; they can also contain sub-directories of files, to any depth). Public Repositories are truly public, as anyone can copy the content. Most will keep upload private, which is where the phrase Pull Request comes in (request the Repository owner to pull in — Merge — some extra and/or changed files).

The key to understanding the interaction between git & GitHub is that git (which is the software) can be used to control a set of files either remotely or locally, and that one of the things that it expects is to handle copy/clone or update operations between two mirror Repositories, one of which is local & the other of which is remote. Typically, the remote Repository is held within GitHub whilst the local mirror may be one of (possibly) hundreds of local mirrors being worked on at any place within the world. There can thus be hundreds of updates to the remote mirror from the distributed local mirrors. In addition, each local mirror will have a schedule of regular updates from remote to local in order to keep up to date with changes from all other users.

Here were my recent actions step-by-step:–

  1. Create an Account on GitHub
      
    This is needed only if you need to update a GitHub repository (no GitHub account is required to retrieve a file from a GitHub public repository).
     
    Go to GitHub and create a username. It needs to be unique.
     
    My OSM username was already taken (dammit) so I originally chose ‘Alexander-Kemp’. Fortunately GitHub has a rename-facility (under Settings | Account), so I renamed to ‘alexkemp9’. Do not do anything like this after you have created a repository.

  2. Fork an Existing Repository
    (see also Forking Projects)
     
    Whilst logged in (that’s the bit that foxed me for a bit) go to the project that you are interested in & click on the Fork link at top-right. fork pic
    After GitHub has finished processing you should be back at your own repository page & able to see the newly-forked repository.

  3. Clone an Existing Repository
    (see also Cloning Repositories)
     
    Halfway down the page on the right-hand-side of a Repository page is a “Clone or Download” button:
    clone pic
    All you will need is the URL that it provides.
     
    This is the relevant page for the repository that I cloned:–
    https://github.com/alexkemp9/openstreetmap-promotional-leaflets
    And the clone URL provided from that page is:–
    https://github.com/alexkemp9/openstreetmap-promotional-leaflets.git
     
    That repository was cloned to my hdd by taking the following actions at the terminal command-line (note that for me git was already installed; go to git downloads if you are on Windows/Mac and need to install it, or use your normal distribution install methods if you work under Linux):–

     cd ~/    
     git clone https://github.com/alexkemp9/openstreetmap-promotional-leaflets.git    
    

    That creates ~/openstreetmap-promotional-leaflets/, drops all files from remote into the directory including ~/openstreetmap-promotional-leaflets/.git + necessary plain-text entries to help make it easy to push/pull files up/down from remote. The .git hidden directory is also the point of cloning — rather than copying — the repository (a full-stop at the start of a file and/or directory name means that it is hidden from ordinary file-listings). Cloning the repository means that you will be keeping the local mirror in step with the remote mirror.

  4. Update the Clone
    (see also Fonts missing from OSM Promotional Leaflets)
     
    (I copied various files across from my earlier work to the clone repository)

  5. Commit then Push the Clone
    (see also How can I upload committed changes)
     
    git needs to be informed of changes to the local mirror before you can push those changes up to GitHub. git maintains a config file in 3 possible places:–
    > a. local<repository>/.git/config — repository-wide values
    > b. global$HOME/.gitconfig — per-user fall-back values for .git/config
    > c. system/etc/gitconfig — system-wide default config

    Here is the bunch of commands that committed all changes to git and then pushed those changes up to remote. The GitHub email address is found at Settings | Emails. git config commands are one-time for each --local (default), --global or --system.

     $ cd ~/openstreetmap-promotional-leaflets
     $ git config --local user.name 'alexkemp9'
     $ git config --local user.email '46583865+alexkemp9@users.noreply.github.com'
     $ unix2dos -n README.md README.txt
     $ git add ./*
     $ git status
     On branch master
     Your branch is up-to-date with 'origin/master'.
     Changes to be committed:
       (use "git reset HEAD <file>..." to unstage)
        
     	modified:   README.md
     	modified:   README.txt
     	new file:   osmflyer.pdf
        
     $ git commit -m "Added osmflyer.pdf (combo of osmflyer?-r15371.pdf)"
     [master e47f9a6] Added osmflyer.pdf (combo of osmflyer?-r15371.pdf)
      3 files changed, 22 insertions(+), 22 deletions(-)
      create mode 100644 osmflyer.pdf
     $ git push origin master
     Username for 'https://github.com': alexkemp9
     Password for 'https://alexkemp9@github.com': 
     Counting objects: 5, done.
     Delta compression using up to 4 threads.
     Compressing objects: 100% (5/5), done.
     Writing objects: 100% (5/5), 16.12 MiB | 601.00 KiB/s, done.
     Total 5 (delta 3), reused 0 (delta 0)
     remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
     To https://github.com/alexkemp9/openstreetmap-promotional-leaflets.git
        bb69dda..e47f9a6  master -> master
    

    git config -e will launch the default terminal editor with the relative config file loaded. Be careful, since that editor may be vim (which is fine if you are a fan of the ’60s or ’70s). Use sudo update-alternatives --config editor to find which is the default, and change it if you wish.

     $ cd ~/openstreetmap-promotional-leaflets
     $ git config -e
     ^K  I    ~/openstreetmap-promotional-leaflets/.git/config (ini)
     [core]
             repositoryformatversion = 0
             filemode = true
             bare = false
             logallrefupdates = true
     [remote "origin"]
             url = https://github.com/alexkemp9/openstreetmap-promotional-leaflets.git
             fetch = +refs/heads/*:refs/remotes/origin/*
     [branch "master"]
             remote = origin
             merge = refs/heads/master
     [user]
             name = alexkemp9
             email = 46583865+alexkemp9@users.noreply.github.com
    
  6. Create a Pull Request
    (see also Pull request from a Fork)
     
    Finally! At the end of this marathon I created a Pull Request.
     
    The main issue is to go to the Repository that the Fork was created from and click on the “New Pull Request” button to the right of the “Branch” menu:
    pull request image

Location: Lace Market, St Ann's, City of Nottingham, East Midlands, England, NG1 1LJ, United Kingdom