Mark Nottingham co-authored this post.
The newly chartered AI Preferences (AIPREF) Working Group will work on standardizing building blocks that allow for the expression of preferences about how content is collected and processed for Artificial Intelligence (AI) model development, deployment, and use.
The use of Internet content for training large language models (‘AI’) has become a contentious topic. Training a model requires voluminous information, and the Internet is readily available as a source of diverse content. However, many publishers and authors object to these new uses of their work without permission or compensation.
While the IETF doesn’t take a position on the legal questions in this space, we do track developments, and last September the IAB AI-CONTROL Workshop highlighted a need for clearer communication between content publishers and AI model trainers.
Right now, AI vendors use a confusing array of non-standard signals in the robots.txt file (defined by RFC 9309) and elsewhere to guide their crawling and training decisions. As a result, authors and publishers lose confidence that their preferences will be adhered to, and resort to measures like blocking their IP addresses.
To address this need, the AIPREF Working Group has been chartered to define:
- A common vocabulary to express author and publisher preferences regarding the use of their content for AI training and related tasks.
- Means of attaching that vocabulary to content on the Internet, either by embedding it in the content or by formats similar to robots.txt, and a standard mechanism to reconcile multiple expressions of preferences.
The Working Group held its first meeting during IETF 122 in Bangkok, where administrative and initial scoping discussions took place. From 8 to 10 April, the Working Group will hold an interim meeting in Brussels, Belgium. This is an open meeting, but registration is required.
If you are interested in monitoring or participating in this work, subscribe to the AIPREF Working Group mailing list.
Suresh Krishnan works as a Distinguished Engineer (Strategy, Incubation and Applications) at Cisco. He has chaired various IETF Working Groups, a research group in the IRTF and has authored more than 30 RFCs across multiple IETF areas.
Mark Nottingham has contributed to developing and maintaining the Internet and web for more than 20 years. He’s written, edited, or substantially contributed to more than thirty IETF RFCs and W3C Recommendations.
Originally published on the IETF blog.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.