In this post, I’ll try to give some insights into the more recent work and workflows of the global State of the Map (SotM) program committee. After having been a member of the SotM program committee for the last couple of years, I figured this might be useful or at least interesting for other program committees or content teams.

Please note that the views and experiences expressed in this post are my own and that they are mostly based on my memory. I’m also trying to just describe how we worked, which may not be the best way, maybe not even a recommendable one, but one that seemed to work for us. Just because it worked for us doesn’t mean it will work for others though, and vice versa.

In some way, this post is also a follow-up to a previous post of mine where I wrote about the software and services behind the State of the Map. I’ll try to avoid duplicating content, so see the previous post for more information about the tools and services that were used.

Organisation

As long as I’ve been on the program committee, it was always organised such that it was mostly split into a ‘core program committee’ (I’ll refer to it as ‘core team’ in the rest of this post) and the full program committee (I’ll refer to it as ‘program committee’ in the remainder of this post), which included all members including the core team. I don’t remember if this split was a conscious decision, if it just emerged based on asking the program committee members about who wanted to help with which tasks, or if there was another reason. But at least so far, we haven’t had a reason to change this.

Program Committee

The primary tasks of the program committee were reviewing and rating the submissions of the talks, workshops, etc. and providing feedback on e.g. the draft for the call for participation before it got published. Once the call for participation was published, the program committee members were also encouraged to announce it in their local communities.

Core Team

In addition to the above, the core team also drafted the call for participation, was responsible for the coordination with the SotM working group (SotM WG), prepared the actual conference schedule/program and communicated with the speakers. Based on this definition, I can be considered to have been a member of the core team.

Phases

The journey from drafting the call for participation to actually doing the conference was comprised of multiple, more or less distinct phases. I’ll roughly describe them in the subsequent sections in a chronological order, but some of the phases also tended to overlap.

Call for Participation

One of the first tasks was drafting and publishing the call for participation. Those who read the call in the last few years probably noticed that it was usually more or less the same content - at least in the most recent years, we usually actually used the call from the previous year as a basis and applied changes to it based on new experience or ideas. This ‘reuse’ isn’t set in stone though, so we might eventually do a more fundamental overhaul. The drafting process itself was usually performed in a collaborative online editor while we were in a conference call or, if I remember correctly, at least once even during a face-to-face meeting.

Review

Once the deadline for the submission of talks, workshops, etc. was over, the review phase started. Reviewing a submission meant giving it a rating and ideally also giving a short, personal, internal comment about the talk. We usually had four possible ratings, namely 0 (“no, does not fit”), 1 (“I don’t like but if you want”), 2 (“good”) and 3 (“excellent, we have to”). The comments were primarily used to make a decision in cases where several talks had roughly the same rating but we could not take all of them because there was e.g. not enough space or too much content overlap between different talks. They were also useful to give further hints, information or options for a talk, which was tremendously useful when making the final decision and creating the actual schedule.

So far, we didn’t have an explicit minimum number of reviews per submission or reviewer, both merely were ‘as many as possible’, which has worked reasonably well.

Acceptance/Rejection of Talks and Creation of Schedule

Eventually, we had to finally decide which talks we accept and which ones we sadly have to reject and how to schedule the accepted ones. For that, the duration of the conference and the number of rooms gave an upper limit for the number of talk slots that were available. Since there were also opening and closing sessions and usually a couple of lightning talk sessions as well as the academic track, the number of talks we were able to accept was actually smaller than this. Once this number was determined, one could basically sort the submissions by the average rating and accept as many talks as we had slots for. In practice, it was a bit more complex than this and we might even have had to reject submissions with a good rating if there was too much overlap with another submission or if there was a ‘conflict’ with the rating criteria that we published as part of the call for participation. This could for example happen if too many people from the same organisation had submitted talks. Submissions with the same or very similar ratings around the ‘cut-off’ also got more attention. These were the cases where the comments in the ratings were especially useful. Sometimes, these comments also lead to a ‘downgrade’ of a regular talk to a lightning talk - or the ‘upgrade’ of a regular talk to the keynote presentation.

After we more or less decided which talks to accept, which ones to reject and updating the state of the talks in the conference planning system (Pretalx) accordingly, the building of the schedule started. The order of these steps probably varied a bit from year to year as did the way we built the schedule: We either did this in a face-to-face meeting or online during a conference call. The first draft of the schedule was either done on a physical or a virtual whiteboard using virtual or physical cards, respectively. These cards contained some information about the talks like the talk title, speaker, rating, track, reviewer comments, etc. Once we were happy with the schedule, we ‘transferred’ it back to Pretalx. If there was a conflict between our draft schedule and the availability of a speaker, we tried to resolve that within Pretalx, which detects these issues. Afterwards, we created a first ‘beta’ release of the schedule and let Pretalx send the notification emails to speakers which informed them about the acceptance or rejection of their submission and about the time when their session was scheduled.

Accepted speakers then had to confirm their session(s), so the ‘official’ publication of the schedule was usually postponed a bit until we got these confirmations. In anticipation of speakers who (had to) cancel their session for some reason, we always kept some ‘backup talks’ - these usually were talks that we would have had to completely reject because their rating was below the cut-off, but which we nonetheless liked or where we assumed that the speaker would likely be ok with that - and where they were most likely going to be attending the conference even though their submission was initially rejected and they might thus only be able to hold their session if another speaker cancels. For these backup talks, we usually edited the rejection notification emails in Pretalx before we sent them and asked the speakers if they would be ok with potentially still doing their talk if we needed them as a ‘backup’.

Speaker Information and “Support”

Especially during the purely virtual SotM conferences, we also had to disseminate quite a bit of information to the speakers, for example how to produce and submit their pre-recorded talks, how the Q&A sessions work, etc. Speakers also might have questions, for example about the infrastructure that is available at the conference venue or some other organisational details. For the dissemination of information, we again used Pretalx which has useful filters for this (i.e. to select the correct recipients from the people known to Pretalx), and questions from the speakers were collected in a ticketing system. This was useful to make sure no message got missed or replied to more than once. Since not all speaker questions were sent to the email address of the ticketing system but to the email address of the SotM working group mailing list instead, we also participated in that list, directly responded to speaker questions there and tried to guide the speakers to the ticketing system. Some of this also required coordination with the SotM working group and the local team. Occasionally, there were also changes to accepted and confirmed submissions, for example, the time when a speaker was available might have changed. In that case, it was then also required to move around some other talks in the schedule, resolve potential new conflicts and inform the affected speakers.

Conference

Once the conference started, the work of the program committee was pretty much done, except for some last minute questions or, unfortunately, the occasional last minute talk cancellation which required at least a schedule update and publication. Apart from that, this was the time where one was finally able to watch all the exciting sessions one saw while reviewing the submissions, enjoy the conference, meet fellow program committee members, old and new friends and of course many other people with an interest in OpenStreetMap.

Since there was no global SotM in 2023, the program committee was also mostly on standby since SotM 2022 in Firenze ended, but we will continue with our work in early 2024 by starting to draft the call for participation at SotM 2024 in Nairobi, Kenya.

After having been involved with the organization of the last few global State of the Map (SotM) conferences, I recently got asked if I could provide some information about the technical side of how hybrid events work to one of the organizers of a future local SotM conference. While writing down some of these details, I figured that this might be of general interest, hence this diary post which also contains various aspects about hybrid events based on our hybrid SotM 2022.

The post will try to cover the last three global SotMs and thus not only technical aspects of a hybrid conference: SotM 2020 was originally planned to take place in Cape Town, South Africa, but had to be moved to a purely online format for well-known reasons. Still for the same reasons, SotM 2021 was planned as an online conference from the beginning. And finally, SotM 2022 in Firenze was our first hybrid conference.

Please note that all views, opinions and experiences expressed in this post are mine and not necessarily shared by the SotM Organizing Committee or other SotM Organizing Committee members. The post is also mostly based on my personal memory only, so I certainly missed or wasn’t aware of many aspects and details, especially in all the areas where I wasn’t involved at all. It also concentrates on the technical side of things and how we did them, not so much on e.g. more social aspects or why we did something in the specific way we did it.

Organization and planning

Much of the planning was done within the GitLab-hosted GitLab of the OpenStreetMap Foundation (OSMF) using tickets, which is useful to keep track of open tasks and for the related asynchronous communication in a distributed team. Our regular meetings were held in the BigBlueButton instance of the OSMF and notes, drafts etc. were usually written using ChaosPads in an instance run by the Chaos Computer Club (CCC). Other internal communication was done using our OSMF-hosted organizing committee mailing list and a Telegram channel.

Attendee information

Most of the attendee information was on the SotM website which was maintained on GitHub with one repository per SotM and built using Jekyll. Some parts of the SotM website, like the conference schedule, were built using custom scripts.

We also used the OpenStreetMap wiki for some information and for example the registration of pre-recorded lightning talks, virtual, self-organized sessions, etc. The registration for on-site lightning talks and self-organized sessions at our first hybrid SotM was done in an analogue way though by using whiteboards at the conference venue again.

Call for participation (CfP)

Like in previous years, we’ve used pretalx to collect, rate and select submissions and to schedule the different sessions. We also used it to send emails with information to the speakers. The sender address of these informational emails was set to be the email address of a queue in the OSMF-run OTRS which was also published on the CfP - so if a speaker replied to our emails or sent us a more general question, it ended up in there and produced a ticket. This is especially useful if there are multiple people who reply to these emails and to keep track of open issues.

We’ve used pretalx as a hosted service, but since it is OpenSource, it can also be hosted on one’s own infrastructure.

Tickets

The ticketing has changed quite a bit during the last three years - if I remember correctly, we didn’t even have conference tickets in 2020, so everybody could just attend by watching the streams, chatting on IRC, writing questions in pads, participating in self organized sessions that were hosted in various video conferencing platforms, etc.

In 2021, we used a conference platform for the first time but gave away the tickets for free. One year later, at our first hybrid conference, we didn’t have free tickets anymore, neither for on-site nor remote attendees, but the tickets for remote attendees were cheaper than those for on-site attendees.

The system we’ve used to give away or sell tickets, respectively, was pretix, which is also OpenSource. We’ve also used it as a hosted service.

Conference platform

In the last two years, we’ve used venueless as conference platform which provided the integrated video streaming, chat, Q&A, polls, BigBlueButton, sponsor ‘booths’ and informational pages. It is mobile friendly and it is also useful for on-site conference attendees, especially during the Q&A sessions. The latter can be a bit awkward though, but has the advantage that one can ask questions at any time during the talk. That way, the (often, but not necessarily local) session host can also handle questions from both local and remote attendees the same way, without unintentionally favoring local attendees over remote ones or vice versa.

At our hybrid SotM, both local and remote attendees therefore got access to venueless with their conference ticket which allowed them to interact with each other in the global chat rooms or the talk specific chat rooms next to the streams where everybody could also ask and rate questions in the dedicated questions tab.

For those who are not familiar with venueless or who want to take a quick look at it, there is at least an English and a German demo instance available. Interestingly, one seems to get more rights in the German instance than in the English one right now, so one can see more of the features in the German demo.

venueless is also OpenSource and could thus also be self-hosted, but we’ve used it as a hosted service, too.

pretalx, pretix and venueless integrate nicely, i.e. one can use the conference schedule from pretalx inside venueless and directly create tokens for access to venueless using pretix.

Video

Video in the form of live video conferences and live streams of the sessions is certainly among the most important elements of a hybrid conference, as it allows remote attendees to see and hear what is presented at the conference. It also lets remote speakers hold e.g. workshops and talks and makes it possible for them to participate in Q&A sessions. The final sections will thus cover various video related aspects.

Remote speakers

We generally asked the remote speakers (i.e. all speakers in 2020 and 2021, but we also had some in 2022) to pre-record their talks since there’s less that can go wrong when compared to holding a live talk remotely. So usually, only introduction and Q&A of a talk with a remote speaker were done live. The only exceptions were some talks in the academic track which were done using BigBlueButton inside venueless in 2021 or the proprietary video conferencing solution provided by the university in 2022. At least for some of these talks, we did have a pre-recorded video as a fallback, so even if there had been last minute technical issues that could not be resolved in time, we wouldn’t have had to cancel any sessions.

The speakers were free to use any application they liked to produce the pre-recorded video, but we did recommend using OBS since it is OpenSource and available for many platforms. Some of us also had prior experience with OBS, so we would also have been able to help the speakers. We also prepared a pre-recorded talk tutorial.

We always used Nextcloud to collect and distribute the pre-recorded videos and other talk assets like presentation slides, in 2020, it was an instance hosted by a member of the local team in Cape Town, the following two years, we used the Nextcloud instance of the OSMF.

In a hybrid setting, the Q&A is a bit of a challenge as it’s hard to get the audio right because of issues with feedback, etc. To avoid such issues, the session host of talks with a remote speaker moved to a different room (where we had a laptop computer with headset prepared), at least for the Q&A part, and joined the same video conference as the speaker. We then showed this video conference in the lecture hall. In this situation, the possibility to ask questions via venueless came in handy: The session hosts were able to ask the questions the local and remote attendees entered into venueless even though they were in a different physical room. It would also be possible to have a remote session host for these sessions, which would even tend to simplify the on-site logistics.

Workshops

Workshop sessions are a bit ‘special’ since they are more interactive than regular talks, so they tend to be slightly more challenging in a purely online or hybrid conference. We therefore didn’t attempt to do any workshops in 2020, in 2021, we did them using BigBlueButton instances inside venueless. Unfortunately, we had some issues with these instances which prevented some attendees from joining them as they didn’t seem to be able to handle as many users as anticipated. At our hybrid SotM, we had workshops again, but split them into purely online (again, using BigBlueButton in venueless) and purely on-site workshops such that speakers didn’t have to constantly switch between ‘virtual’ and ‘real’ world. We offered the speakers to choose if they wanted to hold the workshop twice though, i.e. once online and once on-site, or only once, either online or on-site. Since speakers who are on-site when they hold their workshop for remote attendees need a quiet space for this, we reserved a dedicated room for them where they could use the WiFi to connect to the video conference.

Streaming

In 2020, the video production, streaming, recording, etc. was done by the CCC Video Operation Center (c3voc) and some volunteers, mostly from the local team in Cape Town and the SotM Organizing Committee. The video production was done on centralized OBS instances and then streamed using the c3voc content distribution network (CDN). The live parts were done in Jitsi and we used Jibri to get the Jitsi conference into OBS.

One year later, we ran our own streaming server and ‘micro CDN’ on rented servers and produced the video with the help of several volunteers who ran OBS on their own computers at home. For the live portions of the conference, like introductions and Q&A sessions, we used the OSMF-hosted BigBlueButton that we recorded from within OBS. The ‘micro CDN’ was used for the public stream on the SotM website. The streaming server basically received an RTMP stream from OBS which it both forwarded to the venueless streaming infrastructure and also re-encoded it in different qualities as HLS for streaming via our ‘micro CDN’. Our infrastructure was setup using Ansible and primarily built around nginx, nginx-rtmp and ffmpeg. It was based on some previous experience and ideas taken from the very helpful c3voc and DebConf video team documentation and Ansible playbooks.

The streaming during the hybrid SotM in 2023 was done in a very similar manner as in the previous year. The main difference was that we’ve used the proprietary video conferencing solution and lecture hall audio/video infrastructure that was provided to us by the university in Firenze to produce the stream (instead of using e.g. OBS and dedicated/rented audio/video infrastructure). The stream was then sent to our streaming server via RTMP again. From there on, the basic architecture was nearly identical, except for several tweaks and extensions which were in part required to cater for the multiple streams we had. We’ve also provided an audio-only stream using Icecast which we didn’t have in 2021 (but which was available on the c3voc CDN in 2020) to cater for people with low internet bandwidth. We also tried to make the presentation slides available from pretalx and our website such that they could at least follow the slides while listening to the audio.

Since the infrastructure in the lecture halls was setup such that the stream was produced directly on the presentation computer, speakers couldn’t use their own laptop computer for the talk, they had to use the presentation laptop we provided instead. As a consequence, we had to get talk assets like slides required during the presentation onto this laptop. Part of the solution for this was telling the speakers to upload their talk assets to pretalx. We then had a script that was running on a server which retrieved the uploaded talk assets and uploaded them to the OSMF Nextcloud from where they got synchronized to the presentation laptop.

Release

The videos recorded during the conference require some post processing and cutting after the conference. In 2020, this was done on the c3voc infrastructure, in the last two years, we also did this ourselves, mostly using Shotcut and a bunch of custom scripts that supported our workflow. Most of these scripts are constructing and executing ffmpeg commands. Intros and outros for the videos have been produced using the intro-outro-generator from the c3voc.

Once the videos were cut, the release to media.ccc.de and YouTube was always done with help from the c3voc and using their infrastructure and tools. The meta data (information like the talk title, description and speaker(s)) for the videos that was used in this process was retrieved from the academic and non-academic track schedules that were exported from pretalx and merged using the schedule_convert script.

Closing words

I hope this post was at least somewhat interesting or useful - or that it maybe even sparked or increased your interest in the State of the Map conference! Unfortunately, there’s no global SotM in 2023, but there are some local SotMs around the world which are organized independent of the SotM Organizing Committee, so consider attending one of those to meet fellow mappers!

Preparations for SotM 2024 will already start this year though, too. The SotM Organizing Committee is also looking for new members, and not just in the areas I covered in this post, there are many more tasks that need tackling to make a SotM happen, so if you’d like to help, please get in touch!