OpenStreetMap

wille's diary

Recent diary entries

Using the new OSMCha feature API endpoint

Posted by wille on 4 November 2018 in English (English)

On my last diary post I talked that we have a new features endpoint on our OSMCha API that makes it easy to the communities to add flagged features to OSMCha. Let's check how you can use this new endpoint.

1. Get authorization

The first step is to open an issue on our GitHub repository and tell us what kind of feature modifications you want to flag.

You'll need to deploy some software to monitor the features or you can just write an osm-compare function.

We will review your application and, if you prefer to run a monitoring software by yourself, we will update your user to allow it to post features to OSMCha.

2. Post features

The new endpoint is https://osmcha.mapbox.com/api/v1/changesets/add-feature/. You must do a POST request with the following content:

{ "osm_id": "4321", "changeset": 1234, "osm_type": "node", "reasons": ["Other reason", "Large building"], "version": 54, "name": "Tall Building", "primary_tags": {"office": "coworking", "building": "yes"}, "note": "details about the suspicion" }

Only the first four fields are mandatory. The other are optional. Any other field you add to the request content will not be saved on the database. Check the list of existing suspicion reasons, if you want to flag features with another reason, just add the name of the new reason inside the reasons field and it will be created when the request is received by the server.


If you need some additional help, open an issue on the osmcha-frontend repository or send an email to the OSMCha mailing list. We will be glad to help you validate more changes in OpenStreetMap!

A new OSMCha version is live

Posted by wille on 25 October 2018 in English (English)

After some months of intense work, we released a new version of OSMCha this week! This release includes two big changes: now OSMCha is compliant with the GDPR and we have a new way of receive and store the features.

We simplified a lot the way we store information about the flagged features. It will allow us to store more features on our database and it will be easier to the communities to add features to OSMCha. The documentation of this new resource is available on our wiki. If you need to monitor some kind of changes on OpenStreetMap and would like to use OSMCha for it, please contact us on our mailing list or opening an issue on our GitHub repository.

The GDPR is an European Union law that has the aim to protect the data and privacy of the individuals. To attend the requirements of the GDPR, now the users need to login on OSMCha to see any information about the changesets. However, after you sign in, you won’t notice any change and it will continue working exactly in the same way.

If you notice some error or have some new ideas for OSMCha, please don’t hesitate in contact us. We hope it will open much new possibilities to improve data validation on OSM.

GDPR changes coming to OSMCha

Posted by wille on 7 August 2018 in English (English)

It's been a long time since our last post about OSMCha... In this period, we have implemented few features in the frontend, nevertheless we have done a lot of work on backend and prepared OSMCha to be compliant with the GDPR.

This backend work was necessary to avoid technical debt, make the system run faster and to have a good base to the future improvements. We updated the OSMCha backend to run with Django 2.0, Python 3 and Django Rest Framework 3.7.7. Other backend dependencies were updated as well.

GDPR Changes

It's not on production yet, but we have already started testing a new version of OSMCha with the modifications required by the GDPR. The API and the frontend will show user metadata only to authenticated requests. It means that, if a request is unauthenticated, the fields 'user', 'uid' and 'check_user' will not be present. Furthermore it will not be possible to filter changesets by one of these three fields. The anonymous requests will not return an error, but the results won't take in consideration these filter fields.

If you use the OSMCha API, our staging server is available to your tests, so you can verify the modifications needed by your software (Be aware that the staging server has only changesets older than 2017-09-26). To make authenticated requests, add to the header: Authorization: Token <your_token>. Your user token is available in the user page of the staging server (soon in the user page of the production too).

The saved filter (AoI) RSS feeds will have a different authentication method that we need to implement yet. The features API is yet exposing user metadata, but very soon we will do a big change to it. Instead of exposing a lot of information about the features, it will return just the id, type and suspicion_reasons of each feature.

The GDPR requires some additional changes in other pieces of software that are used by OSMCha. The osm-comments-api and the changeset-map will be closed for public use and available only to OSMCha and some other softwares that are GDPR compliant.

We are planning to put the GDPR compliant version of OSMCha on production on August 27th. If you have some issue or need some help with the changes in the API, don’t hesitate on contacting us by our mailing list or opening issues on github. The new API documentation is available at https://osmcha-django-staging.tilestream.net/api-docs/.

Talk at State of the Map

I presented a talk in State of the Map 2018 in which I spoke about how we are improving OSMCha for the OpenStreetMap community. It was amazing to meet with a lot of OSMCha users and receive new feedback and ideas. You can check the slides or watch the video on YouTube.

Palestras sobre OpenStreetMap em Salvador

Posted by wille on 7 May 2018 in Brazilian Portuguese (Português do Brasil)

Temos muitas atividades do OpenStreetMap em Salvador (e também em São Paulo) durante o mês de maio! Na próxima quarta-feira, 09 de Maio às 16h30, vou falar sobre OpenStreetMap no 1º Encontro de Dados Abertos da Bahia.

Na semana seguinte, dia 18 de Maio, darei uma palestra na Campus Party Bahia, será dia 18 de Maio, às 1h (isso mesmo, na madrugada! :)

Semana que vem também teremos atividade no MundoGeo Connect, dia 16/05 às 11h, mas dessa não irei participar.

Quem perdeu o webinar realizado semana passada, também em parceria com o MundoGeo, pode assistir ao vídeo no YouTube.

Location: Centro Administrativo da Bahia, Salvador, Microrregião de Salvador, Região Metropolitana de Salvador, Mesorregião do Recôncavo baiano, Bahia, Região Nordeste, 41745-005, Brasil

OSMCha News - February/2018

Posted by wille on 9 February 2018 in English (English)

Hello! We just released a new version of OSMCha and we have some new features.

Changeset comments

Based off user feedback, we’ve made changeset comments from OSMCha more flexible.

discussions tab

You can choose to post comments before or after review the changeset, or not at all. You can also set comment templates, so you will have a base text that you can improve before posting. The Review Comment Templates can be edited in your OSMCha user page. We are working to improve this feature, adding the possibility to switch to the discussions tab right after the review is set.

user page

New Location filter

Now we have a new Location filter field. You can define your area of interest by searching for a place:

search place for location field

or drawing a polygon on the map:

draw polygon for location field

The Bounding Box (BBox) filter field is yet available. You can use the BBox size bound in combination with the Location or Bounding Box fields. For example, if you set the BBox size bound to 3, the search will return only changesets whose bbox area is not bigger than 3 times the area of the bbox or location geometry you set on your filter. That way you exclude very large edits that were not restricted to your area of interest.

Map style toogle

We fixed an old bug in the map style toggle. Now you can switch freely between the satellite, streets and dark imagery layers in the changeset visualization.

Roadmap and contact

We have published a roadmap for OSMCha. Do you have some new ideas for OSMCha? You can participate in the OSMCha development by subscribing to our mailing list or opening an issue or pull request in our repository.

Thank you, Amisha and Tridip for the support in this release!

OSMCha News - January/2018

Posted by wille on 9 January 2018 in English (English)

It’s a new year and we have some new features in OSMCha!!! Let’s take a look on them:

Save filter UI improved

save filter instructions

Now it’s easier to save a filter! After set your filter parameters, click on the Save button on the top of the screen, give a name to it and press the Confirm Save button. The filter will be saved and applied, so the sidebar will update with the results. If you want to change something in your filter later, repeat those steps. Remember that the filters have a RSS feed. Get the link in your OSMCha user page.

Changeset comments

Changeset comments example

From now on, when we review a changeset in OSMCha, a comment will be posted in the OSM changeset page. That way, we give a feedback to the user and make that information accessible to other Quality Assurance tools. The comments have a predefined message and are posted with your username.

Tag changes summary

tag changes summary

Each changeset now has a summary of tag changes. This makes it easy to identify the modifications made in the features without the need of clicking on them one by one.

We’d love to hear your comments and suggestions. You can do so by participating in our mailing list or open issues on our github repository.

OSMCha Mailing list

Posted by wille on 21 December 2017 in English (English)

This year has been very special for OSMCha! We have implemented a new interface, a lot of new features and a new backend architecture. We have some new ideas to improve OSMCha in 2018, but we want to start by achieving a better integration with the OpenStreetMap community. So we are starting a mailing list for OSMCha.

As we have more and more people and projects dependent on OSMCha, using the API, deploying their own instances, it is important to have a way to communicate with more agility. A mailing list will be vital to ask general questions, make announcements regarding OSMCha, collaborate on plans and new features, and inform the group of changes.

You can subscribe to the mailing list on https://lists.openstreetmap.org/listinfo/osmcha-dev. Bug reports and wishlist items should continue be reported as Github issues. Please be mindful of the etiquette rules defined on OSM Wiki.

Look forward to talking OSMCha development with you all!

#CompleteTheMap Brasília - o evento

Posted by wille on 6 December 2017 in Brazilian Portuguese (Português do Brasil)

Complete The Map Brasília

O #CompleteTheMap Brasília é um desafio proposto pela Mapillary para melhorar o mapa de Brasília. Todos podem participar do desafio através de coleta de fotos nas ruas e de edições no OpenStreetMap (um projeto de mapeamento colaborativo com mais de 4 milhões de colaboradores em todo o mundo). No dia 13/12, quarta-feira, realizaremos um evento para falar sobre as possibilidades trazidas pela plataforma de computação visual da Mapillary e sobre como a colaboração pode tornar Brasília uma cidade ainda melhor.

O evento começará às 19h e será realizado no Manifesto Coworking, na CLN 206 - Bl A. Teremos alguns brindes para os presentes. A participação é gratuita. Inscreva-se!

Reviewing new users edits in Brazil

Posted by wille on 2 December 2017 in English (English)

Yesterday I created a filter in OSMCha to get all edits made by new users in Brazil, as we are having a new wave of Pokemon motivated edits. Today I spent some time reviewing some of those edits. I sorted the changesets by the number of features created and started by the big ones.

I reviewed 11 edits, six of them had between 132 and 400 elements created, and 5 with 400 to 510. It was a surprise to me that only ONE of those edits was bad (the user created a river that doesn't exist). Some edits had some small errors of course, but nothing very dangerous to the map quality.

So, believe in new users, they also can do amazing things and congratulate them by their good edits!

Nova camada de ruas do IBGE para Salvador

Posted by wille on 16 November 2017 in Brazilian Portuguese (Português do Brasil)

Salvador é uma cidade bem difícil de se mapear. A cidade cresceu desordenadamente, com uma quantidade imensa de morros, favelas, ruas estreitas, becos, vielas... Além disso, o relevo é muito acidentado e faltam dados públicos.

A camada "IBGE - Mapa de Setores Urbanos" ajudava a obter nomes de ruas em alguns locais, porém o alinhamento tá bem ruim em muitos locais e faltam muitos nomes de ruas.

exemplo camada

Assim, há alguns dias eu criei uma camada com nomes de ruas da cidade de Salvador/BA, utilizando como fonte o dataset de Faces de Logradouro do IBGE. Essa camada não é perfeita (já percebi alguns erros de traçado de ruas e muitos nomes duplicados), porém, no geral, tem uma qualidade bem superior à antiga camada.

Esse recurso já está disponível no index de camadas do iD e JOSM. Veja onde você encontra a camada no iD:

ativar no iD

Quero expandir a camada para Lauro de Freitas e talvez alguns outros municípios da Região Metropolitana de Salvador. Caso alguém queira ter esse recurso para alguma outra cidade cuja camada de Setores Urbanos do IBGE está com má qualidade, me avise que posso fazer.

Em breve, pretendo publicar o código fonte que usei para tratar os dados e enviá-los para o Mapbox Studio.

Os dados das faces de logradouro do IBGE são licenciados em Domínio Público.

Location: Barra, Salvador, Microrregião de Salvador, Região Metropolitana de Salvador, Mesorregião do Recôncavo baiano, Bahia, Região Nordeste, 40140110, Brasil

Encontro OSM + Mapillary Brasília - Novembro/2017

Posted by wille on 9 November 2017 in Brazilian Portuguese (Português do Brasil)

Desafio Complete the Map Brasília

No próximo domingo, 12 de Novembro, realizaremos mais um encontro da comunidade OpenStreetMap Brasília! Agora reuniremos também colaboradores do Mapillary, já que foi lançado o Desafio Complete the Map Brasília.

Nos encontraremos na Torteria di Lorenza, da CLS 302, na Asa Sul. Nos encontraremos a partir das 15:30. A proposta é tomarmos um café, conversarmos sobre OpenStreetMap e sairmos de carro pela cidade para fazer fotos pro Mapillary. Dependendo no número de pessoas, podemos fazer um grupo para fazer fotos caminhando ou de bicicleta.

Quem tiver interesse em participar, nos avise deixando um comentário aqui ou entrando em nosso grupo no Telegram.

​​Introducing OSMCha API

Posted by wille on 12 October 2017 in English (English)

Recently we released the new version of OSMCha, an application to help the OpenStreetMap community to review the changesets in the map. In this new version, the frontend was rewritten, we changed the backend to serve the data as a REST API and we have added some new interesting features. Let’s talk about some of the new possibilities that came in with these changes.

The REST API allowed us to build a faster and more efficient frontend and it opened up avenues for other applications being able to use OSMCha data. Yes, It’s now possible to build a JOSM plugin to update the status of a changeset or a feature in OSM. The API documentation to make this happen can be found here → https://osmcha.mapbox.com/api-docs/

The main API endpoint is the /api/v1/changesets/ as it allows us to get changesets. It accepts many filters and one ordering parameters. We also have some sub-urls, like /api/v1/changesets/checked/, /api/v1/changesets/unchecked/, /api/v1/changesets/suspect/, etc. That makes it easy to filter the changesets by some boolean fields and accepts filters parameters too.

Filters & Areas of Interest (AoI)

The new version of OSMCha comes with many additional options of fields to filter changesets that weren’t present in the previous version. Almost all filter options are available in the frontend, but you can also check the API docs to verify if you can benefit from some special API capability. One resource that is not still unavailable in the frontend is the possibility to filter changesets by using any geometry type you want, not just limiting to a bbox.

Furthermore, now you can set a filter query and save it as an Area of Interest (AoI), that way you won’t need to set the query parameters again, all you need is to access your AoI URL. Each AoI also has a GeoRSS feed that you can use to be notified for the new edits. You can also easily share your AoIs with other users by sending them the URL.

To save an Area of Interest, make a POST request to its endpoint with the name you want to give to your AoI and the filter parameters, which are the same that we use to query the changesets. The API also supports saving an Area of Interest with any geometry type you want.

Statistics

Do you need stats about an AoI, a user or about a changeset query? We provide it with some endpoints: /api/v1/stats/ gives us stats about the changesets (total number, quantity of harmful, checked and quantity by suspicion reason and by tag). This endpoints supports the filter params. We have the same stats to an Area of interest in /api/v1/aoi/{id}/stats/ and finally the stats of a user in /api/v1/user-stats/{uid}/.

Protection rules and documentation

The API has a throttling mechanism that limits the number of requests by user by minute to avoid our database of being misused. There are some endpoints, like the ones that add and remove suspicion reasons, that were made to help with administrative issues like fix a wrong detection and whose access is restricted to the admin users.

So checkout the documentation and use OSMCha to monitor your areas of edits! If you have some suggestion, feedback or ideas, post a comment or open an issue in github. It will be great to have new insights from the OSM community!

Encontro OSM Brasília - Agosto/2017

Posted by wille on 8 August 2017 in Brazilian Portuguese (Português do Brasil)

osm cake

Vamos realizar mais um encontro da comunidade OpenStreetMap Brasília! Dessa vez, uma edição especial no dia do Aniversário do OSM.

O ponto de encontro será no Vó Ía Café & Empório, localizado na quadra CLSW 301, bloco C, no Sudoeste. Estaremos lá no domingo (13 de agosto) a partir das 16h para tomar um café, conversar sobre OpenStreetMap e mapear as redondezas.

Quem tiver alguma dúvida, deixe um comentário ou entre em contato conosco em nosso grupo no Telegram.

Location: Sudoeste e Octogonal, Região Integrada de Desenvolvimento do Distrito Federal e Entorno, Distrito Federal, Região Centro-Oeste, 70673-207, Brasil

OSM no OpenDataDay Brasília 2017

Posted by wille on 1 March 2017 in Brazilian Portuguese (Português do Brasil)

OpenDataDay

No próximo sábado, 4 de Março, tem OpenDataDay em Brasília! O evento será realizado no Edifício Parque Cidade Corporate, das 14 às 18h. A comunidade OpenStreetMap estará lá para mapear, ensinar quem estiver a fim de aprender e trabalhar em alguns dados públicos que podemos importar no OSM.

O OpenDataDay ocorre anualmente em diversas cidades do mundo. Participei no evento em Brasília no ano de 2015 e foi bem interessante. Será uma ótima oportunidade de nos divertimos e levarmos o OpenStreetMap ao conhecimento de mais pessoas.

Para participar do evento em Brasília, é necessário colocar o nome na lista para ter permissão de entrada no prédio, para isso, acesse: https://pad.okfn.org/p/opendataday2017Brasilia

Data: 4 de Março, 14-18h.

Local: MinC - SCS, Qd. 9, Ed. Parque Cidade Corporate, Torre B 12º andar - auditório

Melhorando o OSMCHA

Posted by wille on 20 February 2017 in Brazilian Portuguese (Português do Brasil)

Este mês eu voltei a trabalhar no desenvolvimento do OSMCHA, um projeto que eu havia iniciado em meados de 2015 com o objetivo de ajudar a detectar edições potencialmente danosas no OpenStreetMap.

A minha motivação ao criar o OSMCHA foi a necessidade que eu sentia de ter uma ferramenta melhor para monitorar as edições que aconteciam no OSM. Era comum eu olhar as estatísticas de quantidade de nós criados, deletados e modificados a cada dia no Brasil e ver alguns picos, com uma quantidade anormal de dados sendo adicionados ou apagados em um dia. Isso me deixava curioso para saber onde esses dados foram inseridos... Houve uma importação? Caso sim, qual a origem dos dados? Alguém deletou dados inadvertidamente?

Além disso, volta e meia encontrávamos edições que diziam claramente no comentário que a origem dos dados foi o google maps ou outra fonte inapropriada. Apesar de a informação estar explícita no changeset, muitas vezes só descobríamos isso depois de muito tempo...

Como é praticamente impossível revisar todas as edições, pensei em criar uma ferramenta que apontasse os changesets com maior potencial de criar danos ao mapa e que analisasse os metadados em busca de palavras suspeitas.

Após a apresentação que fiz no SotM Latam 2015 em Santiago do Chile, a equipe de dados da Mapbox se interessou pelo OSMCHA e passou a utilizá-lo em seu dia a dia. Além disso, a equipe da Mapbox desenvolveu muitos novos recursos e agora estou muito feliz em anunciar que a Mapbox está patrocinando meu trabalho nos próximos meses para melhorar o OSMCHA.

Nas últimas duas semanas, nós melhoramos a documentação, escrevemos mais testes, limpamos o código para descartar algumas partes que não estamos utilizando e preparamos o software para receber as melhorias que estamos planejando.

A principal mudança que teremos no OSMCHA será a interface: em vez de a homepage mostrar uma lista de changesets, iremos apresentá-los diretamente sobre um mapa. Temos também algumas ideias para facilitar o monitoramento de áreas de interesse de cada usuário e ferramentas de notificação, tornar a análise mais inteligente, entre outras. Porém, sobretudo, queremos escutar sugestões da comunidade OSM, para que possamos criar algo ainda mais valoroso para vocês. Caso você ainda não tenha utilizado o OSMCHA, teste-o em http://osmcha.mapbox.com/. Deixe seus comentários aqui ou abra um ticket no Github.

Outros posts sobre o OSMCHA:

Improving OSMCHA

Posted by wille on 20 February 2017 in English (English)

This month I came back to work on development of OSMCHA, a project that I started in the middle of 2015, with the aim of helping to detect potentially harmful OpenStreetMap edits.

My motivation to start OSMCHA was that I felt that we needed a better tool to monitor changes made to OSM data. Looking at statistics of how many nodes were created, modified and deleted in a day in Brazil, it was common to see some peaks in the number of created or deleted nodes. This made me curious to know - where are those changes... Has an import been made? What is the source of the data? Has someone deleted an entire town?

Furthermore, sometimes we discovered changesets that explicitly said that the data came from Google Maps or some other inappropriate source. Despite that information being expressed in the changeset metadata, it was common for those edits to remain in the OSM database for months or years without anybody seeing them.

As it is impossible for the community to review all edits, my idea was to create a tool to point the changesets that potentially can damage the map and to analyse the metadata searching for suspect words.

I showed OSMCHA for the first time in the SotM Latam 2015, in Santiago, Chile. The Mapbox data team became interested and started using it. Moreover, they developed some new features and now I'm very excited to announce that Mapbox is sponsoring my work in the next few months to improve OSMCHA.

In the last two weeks we enhanced documentation, lifted test coverage, cleaned up the code and prepared the code-base to receive the updates we are planning. The main move we are planning to OSMCHA is to rebuild the interface and make it map-based (instead of showing the changesets on a list, we want to exhibit it over a map). We have some ideas to make it easier for users to review the edits in their area of interest, have some notification features and improve the analysis of changesets, among others...

Nevertheless, we want to listen the suggestions of the OSM community to build something better. If you have never used OSMCHA, try it on http://osmcha.mapbox.com. You can post your feedback and ideas here or open an issue on our github repository.

Finally, I would like to thank Mapbox for the support, specially Sanjay Bhangar, who since the beginning was excited about OSMCHA and believed in the potential of the project!

Read other posts about OSMCHA:

Encontro OSM Brasília - Janeiro / 2017

Posted by wille on 24 January 2017 in Brazilian Portuguese (Português do Brasil)

placa

A comunidade OpenStreetMap de Brasília vai se reunir pela primeira vez em 2017! Dessa vez, iremos à região do Jardim Botânico. O ponto de encontro será a padaria Doce Pão, no próximo domingo (29/01) às 16h.

Vamos mapear os pontos de interesse da região, bater um papo e fazer um lanche. Se você nunca mapeou, será uma ótima oportunidade de aprender. Usaremos diversas técnicas como anotações em papel, fotos e aplicativos de celular. Se você já mapeia, vai poder conhecer outros mapeadores e ter um ótimo momento de integração à comunidade.

Também temos um grupo no telegram da comunidade OSM brasiliense: https://t.me/osmbsb.

Location: Jardim Botânico, Região Integrada de Desenvolvimento do Distrito Federal e Entorno, Distrito Federal, Região Centro-Oeste, Brasil

Vídeos e fotos do State of the Map Latam 2016

Posted by wille on 16 December 2016 in Brazilian Portuguese (Português do Brasil)

foto sotm latam

Quem não participou do State of the Map Latam 2016, já pode conferir os vídeos de todas as palestras realizadas no auditório principal do evento. A playlist com 20 vídeos está disponível no YouTube. Tivemos palestras em Inglês, Espanhol e Português.

Temos também várias fotos já publicadas no Flickr.

Almoço OSM Brasília - Setembro/2016

Posted by wille on 28 September 2016 in Brazilian Portuguese (Português do Brasil)

Almoço OSM BSB

Em Brasília, aproveitamos todas as oportunidades para nos reunirmos e batermos um papo sobre o OSM. Dessa vez, recebemos a visita de Narcélio, que veio do Ceará, e nos encontramos para almoçar!

Inscrições abertas para o State of the Map Latam 2016

Posted by wille on 1 September 2016 in Brazilian Portuguese (Português do Brasil)

logo SotM Latam 2016

Em Setembro de 2015, mapeadores e usuários do OpenStreetMap (OSM) se reuniram em Santiago do Chile para três dias de palestras e debates sobre os mais diversos temas relacionados ao projeto. Com a participação de cerca de 150 pessoas, nascia o State of the Map Latam, a conferência latino-americana da comunidade OpenStreetMap.

Este ano é a vez de São Paulo sediar o evento, o qual ocorrerá entre os dias 25 e 27 de novembro. O objetivo é o mesmo da primeira edição: reunir as pessoas envolvidas com o OSM em toda a América Latina, além de empresas e instituições governamentais. A comunidade brasileira é uma das maiores do continente, sendo assim esperado um grande público para a conferência.

A programação será composta por palestras, workshops, grupos de trabalho e de uma maratona de mapeamento, atividade na qual os colaboradores focam em melhorar o mapeamento de algum local ou de um aspecto específico do mapa.

As inscrições para o State of the Map Latam são gratuitas e podem ser realizadas pelo site http://state.osmlatam.org/pt/. Além disso, a chamada de trabalhos está aberta até o dia 25 de Setembro.

Location: República, São Paulo, Microrregião de São Paulo, Região Metropolitana de São Paulo, Mesorregião Metropolitana de São Paulo, São Paulo, Região Sudeste, 01038-100, Brasil