The IETF recently completed its first annual community survey. A summary of the results, originally published on the IETF website, is republished here with permission.
In May of this year the IETF Administration LLC (IETF LLC) on behalf of the Internet Engineering Steering Group (IESG) and in collaboration with the Internet Architecture Board (IAB) distributed the first annual IETF community survey to all 56,000 addresses subscribed to IETF mailing lists. Its purpose was ‘To help better understand our community and its makeup, gather views on the IETF and how well it works for participants, and gain insight into how we compare to similar organizations’.
The key findings from the survey are presented below. We recommend reading the full report, which includes the results of each question presented as charts and commentary, and which explains the methodology and analysis that leads to these findings.
Key findings
Organization
The IETF mostly delivers its mission and principles
From Q25 we learn that the quality and relevance of RFCs is good whereas the focus on the most important work is minimally acceptable, and from Q25 and Q27 that the support for consensus is strong. Q26 and Q27 differ on the ability to share views, with it judged as minimally acceptable for the IETF in general but good for Working Groups (WGs).
IETF processes are mostly acceptable but major improvement is needed
Q25 tells us that IETF processes have minimally acceptable effectiveness and similarly Q26 tells us that the ease of finding information, understanding it, and the quality of that information is acceptable. Q27 goes into more detail on WG processes and those mostly range from minimally acceptable to good.
The standout exception is the slowness of the entire IETF process, judged as poor in both Q25 and Q27.
The IETF has an issue with mass communication and how (or why) to reach the full IETF community
From Q21 we learn that we have a bootstrapping problem with mass communication given the lack of knowledge about ietf-announce and a problem with its content for those that do. From Q22 we learn that alternative communication mechanisms to email announcements are much less preferred now, but are preferred by younger participants.
People
The IETF has a sense of community, but only just and not one that people can recommend
From Q26 we learn that a minimally acceptable number feel part of the community, feel treated the same as the rest of the community, feel that the IETF communicates with them well and from Q27 that behaviour is minimally acceptable, contributions are valued and that WGs are a good use of time.
From Q10 we learn that people are unclear about what participation in the IETF actually means.
Q20 (on how likely respondents are to recommend participation) uses Net Promoter Score scoring and our result shows that the IETF has a poor score for promoters minus detractors of -24.31%.
The IETF has a big problem with gender, a smaller problem with regionality but no problem with age
From Q4 we learn that females make up between 10.7% and 9.2% of the IETF community, significantly lower than the percentage of the population or the percentage split of IT workers, and from Q11 and Q18 that this percentage has not grown in some time. We learn from Q18 that females are disproportionately deterred by the culture of the IETF, from Q26 that they are less likely to feel treated the same as the rest of the community as males, and from Q27 that they are less likely to feel their contributions are valued and that the behaviour of other participants is acceptable.
From Q2 we learn that the US/Canada and Europe still dominate IETF participation and from Q11 that this is only slowly changing. Female representation from Europe and Africa is notably lower than from other regions.
From Q3 we see a good age spread for participants. Q25, Q26 and Q27 show minor differences by age with some lower scores for the 18-24 age range but nothing more.
Participation
The principle of people participating for the benefit of the Internet is true but not the strongest factor
RFC 3935 states that our participants and our leadership are people who come to the IETF because they want to do work that furthers the IETF’s mission of “making the Internet work better” and this is partly supported by Q12 which puts “making the Internet work better” as the second highest reason and with high support. The highest reason is personal interest and the third is professional development, pointing at a much stronger personal benefit than otherwise understood.
The foundations of people participating as individuals is very complex and only partly supported
RFC 2418 states that participation is by individual technical contributors, rather than by formal representatives of organizations and this is also partly supported by Q12 with only 30% reporting that they participate because their job requires them to, implying that the other 70% could choose not to.
The same question also tells us that 48% participate because their job allows them to, Q13 tells us that only 33% of all time spent on the IETF is personal time, and Q18 gives two of the top three reasons why people cease participating as work related. This indicates that people use their autonomy to choose what work they do (and possibly what employer they work for) to favour the IETF. What the survey did not address is what obligation they have to advocate for their employers interests if they choose to participate.
The two pillars of email and English have very strong support
From Q22 and Q24 we see high support for email announcements and mailing list interactions, significantly higher than the presented alternatives, and from Q23 email is the preferred method for providing feedback to IETF leadership. All three results have a tight clustering of support from all age groups, with support slightly stronger the older the respondent.
From Q5 we learn that the general skill with English is excellent and from Q26 that its individual suitability for full participation is also excellent. These are the only excellent scores in the whole survey.
Support for new modes of participation is limited and age dependent
From Q24 we learn that support for new modes of participation is limited in comparison to email and that there is a strong inverse correlation between age and strength of support. Future surveys will tell us if people maintain these preferences as they age and if the IETF should therefore be planning for this change.
Environment
The IETF is a very important part of a well connected ecosystem
In Q28, which uses Net Promoter Score, we get a ‘Great’ score for the number that think the IETF is important for the development of the Internet. From Q29 we learn that 44% of respondents participate in one or more other standards setting organizations and from Q30 that the IETF is rated ‘very favourably’ in comparison to those other organizations.
Next steps
The results of this survey will be used by the IESG and IETF LLC over the next year as they plan and carry out their work. In particular, the survey will be regularly referenced in decision making to ensure a data-driven approach is taken that emphasizes addressing areas that the evidence shows are high priority concerns reducing time spent on those the evidence shows are not actually concerns.
This survey will be repeated annually, with some adjustments, in order to build up a time series of data and to see whether the high priority concerns are being addressed. It is likely that some cleaning up of mailing lists will be undertaken before then, leading to a smaller total population and more accurate results.
Finally, thank you to all those who took part in this survey and provided us with such valuable feedback. Please raise any questions or feedback on the admin-discuss@ietf.org mailing list.
Jay Daley is Executive Director at the IETF
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.