OpenStreetMap

Google doesn't index localized wiki pages

Posted by matata on 18 December 2009 in English.

When I tried to see if Google index the Arabic wiki pages, unfortunately I found that Google doesn't do it.
Try with "maps" in Arabic "خرائط" and filter the result by "site:openstreetmap.org". "http://www.google.com/search?hl=en&q=site:openstreetmap.org+خرائط"
There's no results from the wiki nor from the Map. the only results founded are from the Diary.

Google doesn't help us to spread the word in the Arabic region.

Is there any solution for that?!!

Discussion

Comment from lyx on 19 December 2009 at 01:03

Actually, they do index the pages, just not very good. Your blog post is now found. Their indexer does apparently not know much about arabic; try searching for خريطة on the site and compare the result with a search for الخريطة
There seem to be a few more bugs in addition to that one though; even the second search does not find e.g. http://wiki.openstreetmap.org/wiki/Ar:Tag:shop%3Dbakery

Comment from lyx on 19 December 2009 at 01:29

For the readers that don't speak Arabic: خريطة means "map" while الخريطة means "the map". In english both searches would supposedly return the same result (actually a few more results for "map" without "the") but they return very different results in arabic; it looks like Google treats both as totally different words.

Comment from lyx on 19 December 2009 at 08:47

After getting a bit of sleep, I've had a closer look. A possible problem is that we don't tell Google which language the localized pages use; as there are many languages using arabic script this might prevent them from using language specific search functions. I'll try and add lang parameters to Ar:MainPage to see if it makes a difference.

Comment from katpatuka on 19 December 2009 at 17:39

After comparing osmwiki with wikipedia I see that for all languages in osmwiki the html header of the pages' source shows

xml:lang="en" lang="en"

while in wikipedia I see the languages set correctly, i.e.

xml:lang="ar" lang="ar"

in the header of arabic wikipedia. Don't know if that's the point for not being indexed by google. If I google "OpenStreetMap hat das Ziel, freie geographische Daten" from the mainpage to test german I find a link to osm - if I search "OpenStreetMap açık kaynaklı olan ve dünyanın her yerinden" to test turkish: nothing.

Comment from lyx on 19 December 2009 at 18:33

Wikipedia has independent websites for each language, so they can localize the wiki framework itself. OSM wiki shares one framework for all language versions, that makes this kind of thing a lot more difficult. I have added a lang="ar" to the main content on Ar:MainPage; lets wait a day or two and look if it makes a difference.

Comment from matata on 19 December 2009 at 22:41

Good catch! in Diary it has the Arabic tags:
xml:lang="ar" lang="ar"
http://www.openstreetmap.org/user/matata/diary/8912

Maybe that's the reason why Google can't see the Arabic pages.

any solution for this?

Comment from lyx on 20 December 2009 at 11:36

Searching for خريطة now finds Ar:MainPage, so declaring the language used for the wiki content appears to help. I also added a language declaration to Ar:Map_Features, but the introduction text on that page is as yet untranslated and my knowledge of arabic is not good enough to translate it myself. It would be good if someone could translate that text segment.

Comment from matata on 20 December 2009 at 12:02

Searching for "خرائط" doesn't return any results! and even for "خريطة" it will return wrong result.
BTW, Google doesn't scan old indexed pages daily. It will take long time to scan it again.

Comment from lyx on 29 December 2009 at 15:30

Searching for "خريطة الطريق المفتوحة" now finds Ar:MainPage on the first page of results, so apparently that page has been scanned again. The OSM start page as well as the wiki Ar:MainPage are now both found when searching for "خرائط", but unfortunately not on the first 50 pages of results. That is caused by not many other web sites linking there, so if you have a chance to do so, please write articles about OSM for your local computer club, university group, newspaper or other interested websites; not only will the readers of your article learn about OSM, but also the OSM website will move to better places in the result lists when your article has a link to OSM and/or the OSM wiki.

Log in to leave a comment