⚠️🛡️ Managing an AI Crisis

A user created explicit, illegal (and gross) content on a genAI startup’s platform. The founders immediately went into crisis mode and called Denzel. We serve as fractional general counsel to multiple genAI startups, so fire drills come up regularly. But this was different.

⚠️ THE THREAT

First, let’s get the obvious out of the way - the immoral, disgusting aspect of this particular online conduct. The nature of the content generated by this user (child sexual abuse material (CSAM) made this more sinister, more than a cybersecurity issue.

Secondly, the legal risk here cannot be understated. While the specific regulatory framework around AI remains in flux, plenty of existing federal and state laws already apply.

Under 2024’s REPORT Act, the publisher of a website faces liability for CSAM to the extent that they knowingly possess, disseminate, store, or distribute it. The penalty for knowing and willful failure to report online sexual exploitation of children has increased from $150K-$300K to $600K-$1M, depending on provider size.

Under Delaware law, directors can be held personally liable for failures to oversee the operations of the company. The mishandling of mature/child content would constitute a failure in oversight that could expose them to serious personal liability (and reputational harm).

Early stage companies tend to (1) have limited cash runway and (2) be actively fundraising in a competitive market. In this context, this gap in platform security threatened to become more than a PR nightmare. Coupled with potential legal liability, it posed a threat to the Company’s very existence.

🛡️ OUR RESPONSE

Report. We helped the client report the incident to the National Center of Missing or Exploited Children.

Corporate Governance. We reminded the directors to follow corporate governance best practices, particularly keeping records of the actions that they took to fulfill their duty of oversight and to ensure their access to information on any material risks (e.g., liability for obscene/child content).

EULA. We designed a new one to warn users that when they activate the platform’s mature content mode, they are legally liable for their creations and that the platform would be monitoring, moderating, and reporting violations to the authorities as necessary.

TOS/Privacy Policy. We recommended additions/updates to the Company’s Terms of Service and Privacy Policy designed to:

(1) reserve the right to monitor and moderate all user content, at its discretion;
(2) expressly prohibit content depicting or intended to depict either minors in a mature context (including self-censored versions) or any sexually violent or abusive content; and
(3) disclose that, besides content monitoring, the Company also uses other tools (content removal, suspensions, bans) to enforce content restrictions.

Fight Tech with Tech. We also recommended exploring and implementing new tools for detecting illegal content.

Disclaimer. The contents of this article should not be construed as legal advice or a legal opinion on any specific facts or circumstances. Your viewing and/or use of the contents of this article do not create an attorney-client relationship with Cadet Legal. The contents are intended for general informational purposes only, and you are urged to consult with counsel concerning your situation and specific legal questions you may have.