Internet-Draft Safe Internet Measurement November 2026
Learmonth, et al. Expires 6 May 2027 [Page]
Workgroup:
Network Working Group
Internet-Draft:
draft-irtf-pearg-safe-internet-measurement-15
Published:
Intended Status:
Informational
Expires:
Authors:
I. R. Learmonth
SR2 Communications
M. Knodel
G. Grover
internet Research Lab

Considerations for Performing Safe Measurement on the Internet

Abstract

Internet measurement is important to researchers from industry, academia and civil society. While measurement of the internet can give insight into the functioning and usage of the Internet, it can present risks to users, such as breaches of their privacy, legal consequences, or disruption of services they rely on. This document describes briefly those risks. It also outlines considerations for researchers to reference when designing internet measurements to ensure that those measurements can be carried out with user safety as a priority.

Note

This document is a draft. It is not an IETF product. It does not propose a standard. Comments are solicited and should be addressed to the research group's mailing list at pearg@irtf.org and/or the author(s).

The sources for this draft are at:

https://github.com/IRTF-PEARG/draft-safe-internet-measurement

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 6 May 2027.

▲

Table of Contents

1. Introduction

Measurement of the internet provides important insights and is a growing area of research. Similarly the internet plays a role in enhancing research methods of different kinds.

Performing research using the Internet, as opposed to an isolated testbed or simulation platform, means that experiments co-exist in a space with other services and end users. Researchers using the Internet as part of a scientific experiment include academic, industry and civil society researchers. This document outlines considerations for such researchers so they may mitigate risks to the safety of users and other affected parties.

It is common today for certain internet measurements to require an ethics review before gaining approval and to report details of that review in the resulting work. Previous work in the area of ethical concerns includes the Menlo Report [Menlo] and its companion document [MenloReportCompanion], ethical concerns for censorship measurement [Jones], and [Reidsma] as an example of putting ethical guidelines into practice in ethics review. Many organizations publish specific guidelines around safe measurements, e.g. [TorSafetyBoard].

The measurement landscape and community is wide and varied. The specific context for Internet measurements should be taken into account by researchers/reviewers and applied appropriately. This document does not attempt to supersede any recommendations such as those cited above, but rather to act as a point of reference for high level considerations with particular emphasis on user consent and data collection. In the growing area of research that includes internet measurement we see this work as part of a larger effort to better equip review boards to evaluate internet measurement methods. Future work may specialize these considerations for specific types or contexts of internet measurement.

2. Scope of this document

The document contains considerations for how to measure the internet while prioritizing user safety. The considerations are particularly germane to projects that involve the generation, collection and/or analysis of traffic from humans. When performing research on a platform shared with live traffic from users, that research can be considered safe only if the risk of danger or injury to users and other affected parties arising due to the research, now or in the future, is no more than minimal or, where it is greater, has been accepted through informed consent by those who bear it (Section 5.2). A measurement poses minimal risk when the probability and magnitude of harm anticipated from it are not greater than those ordinarily encountered in everyday use of the internet. Avoiding harm is necessary but not sufficient: safe research also requires that any remaining risks are justified by the benefits of the research (Section 5.1.1), that users have given consent appropriate to those risks (Section 5.2), and that researchers are transparent and accountable for how the research is conducted (Section 5.1.2).

3. Terminology

Threat model: A threat is a potential for a security violation, which exists when there is a circumstance, capability, action, or event that could breach security and cause harm [RFC4949].

User: For the purpose of this document, an internet user is an individual or organisation who uses the internet to communicate, including those whose data is carried in communications over the internet. In certain contexts, users also include individuals who are not the direct subject of the measurement, for example a contact whose data appears in a measured communication.

Affected party: Users, and any other individual or organisation that may be harmed by a measurement, such as the operators of infrastructure that carries, receives or responds to measurement traffic.

Active measurement: Active measurements generate or modify traffic.

Passive measurement: Passive measurements involve the observation of existing traffic without active intervention.

On/off-path: A measurement that is on-path happens on the network. Off-path indicates activity in a side-channel, end-point or at other points where the user, their connection, or their data can be accessed.

One-/two-ended: A one-ended measurement is like a probe or a trace. A measurement with two-ended control requires the cooperation of both endpoints, which might include the network itself if that is the measurement target.

4. Potential user impact from measurement studies

Any conceivable internet measurement study might have an impact on an internet user's safety, including users who are not subjects of the study, and on other affected parties such as infrastructure operators. The measurement of generated traffic may also lead to insights into other users' traffic indirectly as well. It is always necessary to consider the best approach to mitigate the impact of measurements, and to balance the risks of measurements against the benefits to impacted users.

Some possible ways in which users and other affected parties can be harmed as a result of an internet measurement study:

Breach of privacy: User privacy can be affected or violated in many ways. The first consideration is at the stage of data collection itself. First-order data, such as name, can distinguish a person. Second-order data, such as IP address, can be used to track behaviour. The second privacy risk is an internet user's data being shared beyond that for which a user had given consent, through accidental, malicious or compelled disclosure. A third privacy consideration relates to the risk of re-identification or de-anonymization of obfuscated or masked user data. [Kenneally] These risks include data about users other than the study's subjects, such as parties to a measured communication.

Inadequate data protection: A scenario where data, either in transit or at rest, lacks sufficient protection from disclosure. Failure to meet user expectations for data protection is a concern, even if it does not result in unauthorized access to the data. This includes cases of improper access control (i.e. people having access to user data who do not need it).

Traffic generation: A scenario where undue traffic is generated to traverse the internet.

Traffic modification: A scenario where users' on-path internet traffic is non-consensually modified.

Impersonation: A scenario where a user is impersonated during a measurement.

Legal: Users and service providers are bound by a wide range of policies from Terms of Service to laws, each according to context and jurisdiction. A measurement study may violate these policies, and the consequences of such a violation may be severe. At the same time, legal requests may compel disclosure of sensitive data to government entities or to courts in lawsuits. [Kenneally]

Unavailability: Users or other entities may rely on the information or systems that are involved in the research and they may be harmed by unexpected or planned unavailability of that information or systems [Menlo].

System or data corruption: A scenario where generated or modified traffic causes the corruption of a system. This covers cases where a user's data may be lost or corrupted, and cases where a user's access to a system may be affected as a result.

Emotional trauma: A scenario where a measurement of or exposure to content or behaviour in an internet measurement study causes a user emotional or psychological harm.

5. Considerations

This section sets out considerations for researchers designing and carrying out internet measurements, to mitigate the risks described in Section 4. They cover general considerations, obtaining consent from users, data collection and sharing, and the impact of measurements on the network. Not every consideration applies to every measurement, and researchers should apply them in the light of the specific risks their study poses.

Where a consideration in this section allows an exception, or where it is unclear whether a measurement poses more than minimal risk, the decision should not rest solely with the researchers who would benefit from it. Refer it to an independent party, such as an institutional ethics review board. Where none is available, as is often the case for civil society and independent researchers, seek review from an independent body, or from peers outside the project. Record the decision and its reasoning, and report it with the results (Section 5.1.2).

5.1. General Considerations

5.1.1. Analyze the risks and benefits

The benefits of internet measurement should outweigh the risks [Menlo] [Kenneally].

Internet measurement studies should generally advance knowledge and create meaningful results. Benefits of internet measurement studies can include but are not limited to analysis of internet topology; understanding the effect or prevalence of new internet technologies, protocols or applications; characterisation or prevention of malware or other security and privacy threats, and much more. [Kenneally]

The risks of internet measurement studies, particularly to user safety, are summarised in Section 4. Consider those risks, but also those stemming from use of auxiliary data (e.g. third-party data sets). Note that while a privacy risk may not be immediately apparent or realisable, in the future increased computing power may then make something possible.

Weighing benefits against risks is not only a matter of comparing total benefit against total risk: the Menlo Report's principle of justice also asks how risks and benefits are distributed [Menlo]. In internet measurement, the risks often fall on specific users, such as those whose traffic, devices or networks are measured, while the benefits go to researchers, operators or the internet community as a whole.

Identify who bears the risks of a measurement and who receives its benefits. Where these differ, and especially where those at risk are vulnerable or cannot practically consent, reduce the risk as far as possible (Section 5.3.1) and explicitly justify what remains. A study is not justified simply because its total benefits exceed its total risks. No aggregate benefit justifies exposing people to more than minimal risk without their informed consent (Section 5.2).

Ensuring that a project's benefits outweigh its risks requires applying the considerations in this document, as well as regular community feedback (particularly from those who will ostensibly benefit or be at risk through the project), proper auditing of the study, and impact assessments.

5.1.2. Transparency and accountability

Have “a general policy of openness about developments, practices and policies with respect to personal data.” [OECD]

Despite best intentions, things fall apart. In the case of any adverse events, researchers should “responsibly inform affected stakeholders.” [Menlo] Contact affected users directly where possible. In all cases, information about the discovery of security vulnerabilities, data compromise, and instances of unauthorized access to information should also be published, once doing so would not itself put users at further risk (for example, once a vulnerability has been fixed).

At a minimum, report the following with the published results of the research: the outcome of any ethics review; the risk analysis (Section 5.1.1); how consent was obtained (Section 5.2); any exceptions taken (Section 5); and any adverse events.

For long-running experiments, also institute a regular accountability and reporting mechanism, such as an annual review by an advisory board. Such regular accountability mechanisms can include an assessment of the experiment's impact on user safety and privacy, analysis of security events, and an evaluation of whether the experiment is meeting its expected goals.

5.1.3. Identification of traffic

Proactively identify your measurement to others on the network. "This allows any party or organization to understand what an unsolicited probe packet is, what its purpose is, and, most importantly, who to contact."[RFC9511]

Example: For a layer 3 IP packet probe you could mark measurements with a probe description URI as defined in RFC9511.

This guideline may be ignored if attribution runs contrary to the purpose of the measurement study or would otherwise render it ineffectual. For example, for an experiment collecting information about local internet censorship, the network may provide different results if the connections can be identified as serving that purpose. A decision not to identify measurement traffic should be reviewed as described in Section 5, and should be disclosed when results from the measurements.

Identifying measurement traffic also helps to reduce the impact of a measurement on others' infrastructure, as discussed in Section 5.4.1.

5.3. Data collection and sharing

5.3.1. Data collection and minimization

When collecting, using, disclosing, and storing data from a measurement, use only the minimal data necessary to perform a task. Reducing the amount of data reduces the amount of data that can be misused or leaked.

When deciding on the data to collect, assume that any data collected might be disclosed. There are many ways that this could happen, through operational security mistakes or compulsion by a state authority.

When directly instrumenting a protocol to provide metrics to a passive observer, see section 6.1 of RFC6973 [RFC6973] for the data minimization considerations enumerated below that are specific to the use case.

5.3.1.1. Collect the minimum amount of data

Collect only that data that is required to perform the study and discard data that is not required.

When performing active measurements, be sure to only capture traffic that you have generated. Traffic may be identified by IP ranges or by some token that is unlikely to be used by other users.

Again, this can help to improve the accuracy and repeatability of your experiment. For performance benchmarking, [RFC2544] requires that any frames received that were not part of the test traffic are discarded and not counted in the results.

5.3.1.2. Store data securely

Data should be stored in a secure location, with appropriate access controls and full-disk encryption if possible. Access to the measurement data should be minimized to only those personnel who require access to perform research.

5.3.1.3. Minimization techniques

For any data collected evaluate which minimization techniques can and should be applied to that data in order to minimize the risk to users whilst still safely providing sufficient data for the measurement to be effective. A range of pseudonymization or anonymization techniques are available [SurveyNetworkTrafficAnonymisationTech]. IP addresses are particularly identifying, as they can often be linked to a specific subscriber, household or device, and to an approximate location. Where the measurement does not require full IP addresses, they should be discarded, truncated or pseudonymized using the techniques below.

5.3.1.3.1. Mask data

Mask data that is not required to perform the task. This technique is particularly useful for content of traffic to indicate that either a particular class of content existed or did not exist, or the length of the content, but not recording the content itself. The content can be replaced with tokens or encrypted.

It is important to note that masking data does not necessarily anonymize it [SurveyNetworkTrafficAnonymisationTech].

5.3.1.3.2. Aggregate data

When collecting data, consider if the granularity can be limited by using bins or adding noise. Differential privacy techniques [DifferentialPrivacy] can help with this.

Example: [Tor.2017-04-001] presents a case-study on the in-memory statistics in the software used by the Tor network.

5.3.1.3.3. Reduce the accuracy

There are various techniques that can be used to reduce the accuracy of the collected data and make it less identifying while still meeting the needs of the measurement.

The use of binning to group numbers of more-or-less continuous values, coarse categorization in modeling, reduction in concentrations of IP address by geography (geoip) or other first- or second-order identifiers, the introduction of noise and all privacy-preserving measurement techniques that allow researchers to safely conduct internet measurement experiments without risking harm to real users [Jansen].

5.3.2. Data Sharing

Further to use of measurement data, data is often shared with other researchers. Measurement data sharing comes with its own set of expectations and responsibilities of the provider. Likewise there are responsibilities that come with the use of others’ measurement data. One obvious expectation is around end-user consent (see "Implied consent" above). Where data obtained from others was collected in ways that raise ethical concerns, disclose this when using it or publishing results based on it. Allman and Paxson [Allman] provide "a set of guidelines that aim to aid the process of sharing measurement data... [in] a framework under which providers and users can better attain a mutual understanding about how to treat particular datasets."

Their guidance for data providers is to:

  • explicitly indicate the terms of a dataset’s acceptable use
  • convey what interactions they desire or will accommodate.

Their guidance for researchers is to:

  • be thoughtful in the reporting of potentially sensitive information gleaned from providers’ data.
  • comply with the indications and interactions of the data providers.

Example: Researchers have obtained network measurement data from more than one provider for purposes of conducting analysis of protocol use on both. Where privacy partitioning techniques are used, the researchers' findings may inadvertently collude to uncover private information about users. Once realised, researchers should mitigate this privacy risk to end users as well as disclosing this result to the data providers themselves.

5.4. Impact on the network

5.4.1. Impact on others' infrastructure

If your experiment is designed to trigger a response from infrastructure that is not your own, consider what the negative consequences of that may be. At the very least your experiment will consume bandwidth that may have to be paid for.

In more extreme circumstances, you could cause traffic to be generated that causes legal trouble for the owner of that infrastructure. The internet is a global network that crosses many legal jurisdictions and so what may be legal for one is not necessarily legal for another.

If you are sending a lot of traffic quickly, or otherwise generally deviating from typical client behaviour, a network may identify this as an attack which means that you will not be collecting results that are representative of what a typical client would see.

One possible way to mitigate this risk is transparency, i.e. mark measurement-related data or activity as such, as also recommended in Section 5.1.3. For example, the popular internet measurement tool ZMap hardcodes its packets to have IP ID 54321 in order to allow identification [ZMap].

5.4.2. Maintain a "Do Not Scan" list

When performing active measurements on a shared network, maintain a list of hosts that you will never scan regardless of whether they appear in your target lists. When developing tools for performing active measurement, or traffic generation for use in a larger measurement system, ensure that the tool will support the use of a "Do Not Scan" list.

If complaints are made that request you do not generate traffic towards a host or network, you should add that host or network to your "Do Not Scan" list, even if the request is automated. A request may be declined only where honouring it would hide the very thing being measured. For example, in censorship measurement, a request could be an attempt to hide the fact that a certain target is unreachable in a particular geography. A decision to decline a request should be reviewed as described in Section 5, and the requester informed where possible.

You may ask the requester for their reasoning if it would be useful to your experiment. This can also be an opportunity to explain your research and offer to share any results that may be of interest. If you plan to share the reasoning when publishing your measurement results, e.g. in an academic paper, you must seek consent for this from the requester.

Be aware that in publishing your measurement results, it may be possible to infer your "Do Not Scan" list from those results. For example, if you measured a well-known list of popular websites then it would be possible to correlate the results with that list to determine which are missing. This inference might leak the fact that those websites specifically requested to not be scanned.

On the other hand there may be benefits to publish your "Do Not Scan" list including the opportunity to reconsider the list over time, invite feedback, ensure research reproducibility, or bootstrapping other measurement projects or tools.

6. Security Considerations

This document as a whole addresses user safety considerations for internet measurement studies, and thus discusses security considerations extensively throughout regarding collection and storage of user data.

7. IANA Considerations

This document has no actions for IANA.

8. Acknowledgements

Many of these considerations are based on those from the [TorSafetyBoard] adapted and generalized to be applied to internet research. Other considerations are taken from the Menlo Report [Menlo] and its companion document [MenloReportCompanion], and research and discussions of the internet measurement and ethics communities.

Comments and contributions from Marwan Fayed, Jeroen van der Ham, Arturo Filastò, Christian Huitema, Tobias Fiebig, Greg Skinner, Oliver Gasser, Craig Partridge, Eric Rescorla and Shivan Kaul Sahib greatly improved this document.

9. Informative References

[netem]
Hemminger, S., "Network emulation with NetEm", .
[RFC2544]
Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, DOI 10.17487/RFC2544, , <https://www.rfc-editor.org/info/rfc2544>.
[TorSafetyBoard]
Tor Project, "Tor Research Safety Board", <https://research.torproject.org/safetyboard/>.
[RFC4949]
Shirey, R., "Internet Security Glossary, Version 2", RFC 4949, DOI 10.17487/RFC4949, , <https://www.rfc-editor.org/info/rfc4949>.
[Tor.2017-04-001]
Herm, K., "Privacy analysis of Tor's in-memory statistics", Tor Tech Report 2017-04-001, , <https://research.torproject.org/techreports/privacy-in-memory-2017-04-28.pdf>.
[Menlo]
Dittrich, D. and E. Kenneally, "The Menlo Report: Ethical Principles Guiding Information and Communication Technology Research", , <https://www.dhs.gov/sites/default/files/publications/CSD-MenloPrinciplesCORE-20120803_1.pdf>.
[MenloReportCompanion]
Bailey, M., Dittrich, D., and E. Kenneally, "Applying Ethical Principles to Information and Communication Technology Research", , <https://www.impactcybertrust.org/link_docs/Menlo-Report-Companion.pdf>.
[DifferentialPrivacy]
Dwork, C., McSherry, F., Nissim, K., and A. Smith, "Calibrating Noise to Sensitivity in Private Data Analysis", , <https://link.springer.com/chapter/10.1007/11681878_14>.
[SurveyNetworkTrafficAnonymisationTech]
Van Dijkhuizen, N. and J. Van Der Ham, "A Survey of Network Traffic Anonymisation Techniques and Implementations", , <https://dl.acm.org/doi/10.1145/3182660>.
[OECD]
OECD, "OECD Privacy Principles", , <http://oecdprivacy.org/>.
[ZMap]
University of Michigan, "ZMap Source Code - packet.c", <https://github.com/zmap/zmap/blob/main/src/probe_modules/packet.c>.
[RFC6973]
Cooper, A., Tschofenig, H., Aboba, B., Peterson, J., Morris, J., Hansen, M., and R. Smith, "Privacy Considerations for Internet Protocols", RFC 6973, , <https://www.rfc-editor.org/info/rfc6973>.
[Jones]
Jones, B., Ensafi, R., Feamster, N., Paxson, V., and N. Weaver, "Ethical Concerns for Censorship Measurement", , <http://conferences.sigcomm.org/sigcomm/2015/pdf/papers/nsethics/p17.pdf>.
[Reidsma]
Reidsma, D., van der Ham, J., and A. Continella, "Operationalizing Cybersecurity Research Ethics Review: From Principles and Guidelines to Practice", EthiCS 2023 workshop, NDSS Symposium, , <https://www.ndss-symposium.org/wp-content/uploads/2023/02/ethics2023-237352-paper.pdf>.
[RFC9511]
Vyncke, É., Donnet, B., and J. Iurman, "Attribution of Internet Probes", RFC 9511, DOI 10.17487/RFC9511, , <https://www.rfc-editor.org/info/rfc9511>.
[RIPEAtlasEthics]
Kisteleki, R., "Ethics of RIPE Atlas Measurements", RIPE Labs, , <https://labs.ripe.net/author/kistel/ethics-of-ripe-atlas-measurements/>.
[Allman]
Allman, M. and V. Paxson, "Issues and Etiquette Concerning Use of Shared Measurement Data", , <https://conferences.sigcomm.org/imc/2007/papers/imc80.pdf>.
[caida]
CAIDA, "Promotion of Data Sharing", , <https://www.caida.org/catalog/datasets/sharing>.
[Kenneally]
Kenneally, E. and K. Claffy, "Dialing privacy and utility: a proposed data-sharing framework to advance Internet research", , <https://www.caida.org/catalog/papers/2010_dialing_privacy_utility/dialing_privacy_utility.pdf>.
[Jansen]
Jansen, R., Traudt, M., and N. Hopper, "Privacy-Preserving Dynamic Learning of Tor Network Traffic", ACM SIGSAC Conference on Computer and Communications Security (CCS '18), DOI 10.1145/3243734.3243815, , <https://dl.acm.org/doi/pdf/10.1145/3243734.3243815>.

Authors' Addresses

Iain R. Learmonth
SR2 Communications
Mallory Knodel
Gurshabad Grover
internet Research Lab