Publication vs Responsible Disclosure

Publication vs Responsible Disclosure

Academic science has a strong norm of openness: publish your findings so others can build on them. This norm has served science well for centuries.

AI complicates this norm. Some findings are dual-use: the same capability that advances research can be misused for harm. When does the obligation to publish conflict with the obligation to prevent harm?

The Academic Incentive

Researchers are rewarded for:

  • Publishing novel results
  • Publishing first (priority matters)
  • Having work cited and built upon
  • Making findings widely accessible

Careers, tenure, funding, and prestige all flow from publication. Restraint is professionally costly.

The Safety Consideration

Some AI research reveals capabilities that could be misused:

  • Jailbreaking techniques that could help bad actors
  • Model architectures that enable dangerous applications
  • Training methods that make harmful AI easier to build
  • Vulnerabilities in deployed systems

Publishing such findings makes them accessible to everyone, including those who would misuse them.

The Tension

The researcher faces competing obligations:

  • To science and colleagues (openness, reproducibility, progress)
  • To society (preventing foreseeable harms)
  • To career (publication is professionally necessary)
  • To institution (reputation, funding implications)

There’s no formula that resolves this tension. Different cases call for different judgments.

Approaches

Publish everything: Trust in openness. Bad actors will find vulnerabilities anyway; better for defenders to know.

Coordinate disclosure: Share with affected parties before public release. Let them patch before publishing.

Redact details: Publish the existence of findings without details needed to replicate harm.

Delay publication: Wait until defenses exist or the finding becomes less dangerous.

Don’t publish: Some findings shouldn’t be released. Accept the professional cost.

Structured access: Share with vetted researchers but not the public.

Each approach has advocates. None is obviously correct for all cases.

The Institutional Gap

Academic institutions aren’t well-equipped for these decisions:

  • IRBs focus on human subjects, not dual-use publication
  • Journals lack security review processes
  • There’s no standard framework for assessing publication risk
  • Individual researchers bear the decision burden

This is a governance gap. The decisions are being made ad hoc, by researchers without relevant expertise, under professional pressure to publish.

Implications

  • Academic norms developed before dual-use AI research
  • Individual researchers shouldn’t bear this burden alone
  • Institutional structures need to evolve
  • The tension is real and won’t resolve itself

Open Questions

  • Who should decide what gets published?
  • How do you assess publication risk for novel findings?
  • Can academic incentives be reshaped to value restraint?
  • What institutions should govern dual-use AI research?

See Also