Google Summer of Code 2018 GTFS Integration Tool Summary

Posted by srikant123 on 11 August 2018 in English (English)

GSoC 2018 with OpenStreetMap comes to end and with it its the end of the most exciting summer I have ever had which have helped me grow and learn a lot of awesome stuff from my mentors who guided me throughout my GSoC period.

Link to main github repository : gtfsintegrate.
Link to google docs where complete step by step documentation and implementation of application is described googledocs.
Feeds were downloaded from: TransitFeeds.

The Three main views of the GTFS Integration application are:

1. Feed entry Module

In this view of the application user is presented with a form which consists of a feed URL field, name field and a frequency field(number of days after which the feed must be downloaded again). Celery tasks are defined which can run every few minutes checking all the feeds and downloading the feeds if the frequency exceeds when compared to current time-stamp(In the current production server celery is not supported so we are looking for alternatives). In the front-end part application shows whether the feed is present or whether the feed is updated to latest version in the Feed Download Status Block.

Feed Form

2. Nodes editing Module.

Nodes editing Module generally consists of three parts.

i. GTFS Stop fields Correspondence Form.

Using this form user defines which fields in GTFS corresponds to which fields in OSM tags. Since every feed is different from one another and depends on the operator how they input data, so using this form correct fields can be sent to JOSM.

Correspondence Form

ii. Name Normalization Form.

GTFS Feeds can consist of stops with abbreviations application replaces some common abbreviations. In this interface the user is provided with a list of stop names which consists of abbreviations and then user can define what they are. These normalized names are sent to JOSM so it is important to normalize names before sending to JOSM.

Conversion Form

iii. Nodes editing Interface.

This interface consists of data shown on map on the right and information of clicked OSM and GTFS stops on the left. Information consists of a result column to show the user about what will go in OSM if the stop is matched. By default the result column will show name and ref from selected OSM stop if it already has those tags. User can add name from GTFS stop if the GTFS feed has more appropriate name for the stop. Node Editor

Adding stops present in GTFS Feed to OSM server

Application also lets user to create and OSM stop by dropping a marker beside a GTFS stop, tool will add name, ref and bus=yes tags to it and send the node to upload it to OSM server.

Node Addition

3. Relation editing Module

Relations editing Module consists of two parts.

i. GTFS Routes.txt Correspondence Form.

Routes Correspondence lets the user define what field in GTFS feed's routes.txt corresponds to what field in OSM. Agency correspondence can also be filled to get data from agency.txt. Some feeds also have extra_data which can also be added to OSM. After user filling all these information application creates route_master xml. In this interface user can also take one of the entry in routes.txt into reference while filling the form.

Relcorr Form

ii. Relation editing and comparison Interface.

Unique itineraries are extracted for each line from routes.txt and are presented on the interface. Clicking a particular line application will download relations from OSM in bbox of the line.

Relation Editing

Matching relation with itinerary to add missing stops in OSM.

If a stop is missing or out of sequence in OSM Relation that can be matched and data is sent to JOSM for further editing

Relation match

I will keep contributing and maintaining the project even after GSoC period.

Finally I have developed an application which varies in some functionality with the functionalities described in the GSoC proposal. Functionalities which are changed while discussing with my mentor Jo are. 1. Before GSoC we decided to let the application find discrepancies but since it the GTFS data varies from place to place and so does the OSM data we decided to let the user find discrepant stops and transfer data to JOSM. 2. We decided to not to implement the functionality of user informing the GTFS Feed operator about errors in GTFS feed as it would be irrelevant with our final goal.

Other things which I along with my mentors have included in the application are

1. Django Rest API Service:

The data stored in the database can be accessed using url host/api/data. Using this REST API I will develop the front end of the application in React JS. It is even possible for other developers to make use of data.

2. Included Django Swagger:

Django Swagger is integrated so that it is easy for other developers to have an idea about how the data needs to be accessed. Swagger UI at url host/swagger/

3. Created a Docker image using which the application can be downloaded.

In the first phase I deployed the application on heroku. Since heroku has a limit of 10k rows for Postgresql my mentors and I decided to create a docker image of the image of the application. I created the docker-compose and integrated so that it can be deployed on any platform.

4. Enabled automatic install using make into the application.

Using make as documented in projects github repository the application can be installed and run locally without any problem or errors (Tested on Ubuntu 17.04).

5. Integrated Travis.

Travis continues integration is implemented in the app to keep track of tests. Currently it shows the result of tests from multigtfs app present in the project.

Future goals for the project.

  1. Implement goals which were not achieved due to less time which are. i.Show list of stops, itineraries which have issues in the GUI, ii. Modify GUI to be more interactive and informative.
  2. Write tests to prevent application from breaking at different stages and control bugs.
  3. Host the application on a stable virtual server.

Comment from Stereo on 27 August 2018 at 20:00

This looks really cool!

How does this deal with cases where the GTFS only has one centroid for the stop point and OSM has all the individual platforms and stop positions mapped?

Is there any Ubuntu-specific stuff in it, or should it run on anything that runs Django and postgis?

Comment from srikant123 on 31 August 2018 at 18:20

Hi Stereo this project dosen't deal with adding missing stops in gtfs for that there is a tool named osmtogtfs. It should run on any platform which has django, postgis, gdal and geos.

Login to leave a comment