It is voluntary. The convention is used in BIG-bench [1] and mentioned in their README here:
> All task files (including README.md and task.json) contain a "canary" string, which should not be edited. This is to prevent benchmark tasks from leaking into web-scraped training data.
It's not universally adopted and of course anyone is free to ignore it.
> All task files (including README.md and task.json) contain a "canary" string, which should not be edited. This is to prevent benchmark tasks from leaking into web-scraped training data.
It's not universally adopted and of course anyone is free to ignore it.
[1] https://github.com/google/BIG-bench#creating-a-json-task