mixing technologies – weekend edition

January 26, 2014

So. It’s late Saturday night, and here I sit in front of my little hackerish Ubuntu notebook. The clock just rang out midnight, meaning I’m rolling with this into Sunday, and I’m typing away while listening to Depech Mode’s “The Best Of” on Youtube that somebody in Russia ripped and uploaded. Yeah.

It’s been a long three or four days since I first wrote about starting out with Open Street Map, and it’s like I’ve been in a long, drawn-out fist fight, where the punches keep coming in from out of nowhere and I can’t land any of my own. That map above, with the southern states in place from Georgia west to Arizona, took me the majority of the time figuring out how to load.

Now, I haven’t been spending all this time on this task; I do have A Real Job. I’d take time off during a lunch and spend about 30 minutes on it, then spend another hour after getting home and after everything was done for the evening before hitting the sack. Even so, I never expected this kind of slow going.

Biggest problem is in the directions. Remember that page that was so helpful with its simple directions? Turns out it was too simple, and here’s why. At one point you need to load a file using the following command (this is for Florida, which is where I started to build my map of CONUS):

osm2pgsql –slim -C 1500 –number-processes 4 florida-latest.osm.pbf

That command loads the latest Florida map into the database. So far, so good. The problem is that if I use the same command to load any other file, it wipes out any other data. That’s when I found out about the “–append” command line switch. Sweet. Just add that to the rest of the commands to load the rest of the states and I’m done. No, not sweet. When run, I’d get the following error message:

failed: ERROR: duplicate key value violates unique constraint “planet_osm_rels_pkey”

I tried all sorts of ways to get around this, including the implied method of importing all my map files at the same time by placing all the file names on the command line, separated by spaces. That didn’t work either. Finally, while cruising through this forum on github I found out you needed to drop the “–slim” command line switch. WTF? When I did that with the Georgia file, all the other files loaded, and when it was all done I had the basic map you see above. I noticed something interesting about the slim switch. If you read the documentation, it seems to imply that using slim is actually faster than not using it. Not on my system. Leaving slim off actually sped things up a bit.

Note that I got so desperate to load anything in CONUS that I downloaded the 5GB map file of the entire continental North America and attempted to load it. I downloaded that late Friday, and started it up around 6pm. It ran through the night, and continued until I killed the whole thing around 3pm Saturday. That’s when I gave Google another stab at trying to find a solution. I did try DuckDuckGo, but it failed spectacularly in trying to find a solution; it gave exactly two (yes, I said two) hits. Google gave me page after page, and it was literally the first entry that gave up the vital clue about the use of slim. Google 1, DuckDuckGo 0.

Other issues I’ve had with my Open Street Map stack is getting it to generate all the necessary tiles at the default resolutions. I’ve been doing it the hard way with “/dirty” on the end of the map graphic. For example, in Chrome, selecting “Open Image in New Tab” gives you a new tab with “http://localhost/osm/5/7/12.png” in the address bar. Modify the URL to “http://localhost/osm/5/7/12.png/dirty” and hit return, and OSM responds with “Tile submitted for rendering“, and after a bit of time it will be rendered. I did this on a lot of blank tiles, then setting Permalink on that section and hitting refresh until it showed up. This is a royal pain, and no amount of searching calls out clearly what tool needs to be installed and/or run to generate a series of default tiles. I will find out how to do this, as my very labor intensive method sucks dead hamsters through a garden hose.

Here’s a few more screen shots showing the detailed maps I have so far.

The browsers used are, from top to bottom, Opera showing Austin Texas (West Sixth Street and the Colorado River), Firefox showing Orlando Florida (I-4/408 interchange and Lake Eola), and the JavaFX in Java browser showing the states of Tennessee, North and South Caroline, Georgia and Alabama. I’ve started to hack the original JavaFX demo application, having reversed the tab order and setting the default URL to point to the OSM slippymap page. The next step is to build a PostgreSQL viewer into the second tab so I can browse the GIS tables that hold the OSM map data. Before I do anything else I’m dumping the database to back it up. I want to add the rest of the continental US states, and maybe a few of the Canadian cities like Toronto.

While I was at it I also pointed my Nexus 7 2012 tablet, running KitKat, at the OSM stack, just to see how it rendered on an Android tablet. I have my reasons. My systems are wirelessly connected so it wasn’t any big deal to punch in the IP of the Ubuntu system along with the default slippymap URL and give it a go.

Both of these screenshots were using Opera. I have to say that I really enjoy using Opera on my Samsung Galaxy S4 (Android 4.3), Nexus 7 2012 (Android 4.4.2) and even my Barnes & Noble Nook HD+ (Android 4.0). It seems to be fast and use a minimal amount of memory. I like the Speed Dial page, especially on the phone. The only problem I had rendering my local OSM map was with Firefox, as you can see below.

I don’t know what’s happening here, but dragging the map around caused it to tear apart. I don’t know why that happens, but I won’t lose any sleep over it. Firefox does just fine with Google Maps and the commercial implementation of OSM, so it’s definitely something in the current installation. Since I need to make all Javascript references on slippymap local, I hope dropping the latest versions of OpenLayers and OpenStreetmap will fix that issue. Speaking of commercial implementation, I came across this courtesy of DuckDuckGo:

Yes, it’s MapQuest Open, but this site is using Open Street Map data as explained here. Looks kinda pretty, don’t it? Anyway, that’s it for the evening. It’s now past 1 AM, and I’ve had enough excitement for one weekend.