SB 735 — AI Innovation Trust Fund.

FILED SENATE GENERAL ASSEMBLY OF NORTH CAROLINA MarSESSION 25, 2025 S.B.S 1 SENATE BILL 735 Short Title:

735 SESSION 2025 PRINCIPAL CLERK S D SENATE BILL DRS35250-MCa-139A Short Title:

Rules and Operations of the Senate March 26, 2025 A BILL TO BE ENTITLED AN ACT TO ENACT THE ARTIFICIAL INTELLIGENCE INNOVATION TRUST FUND.

The fund shall consist of appropriations to the Department of Commerce to be allocated to the fund, interest earned on moneymoneyin in the fund, and anyanyother other grants, premiums, gifts, reimbursements or other contributions received by the State from any source for or in support of the purposes described in this subsection.

Funds in the fund are hereby appropriated to the Department forthepurposes set forth in this section,and, except as otherwiseexpresslyotherwiseexpresslyprovided, provided, theprovisionsofthissectionapplytopersonsreceivingagrantorassistance fromthefund.

(ii) undermining or impeding critical aspects of a mission, *DRS35250-MCa-139A**S735-v-1* General Assembly Of North Carolina Session 2025 program or organization;

A covered model or covered model derivative autonomouslyautonomouslyengaging engaging in behavior other than at the request of a user.

– The legally responsible organization, corporation, or entity that directlydirectlyoversees oversees and controls the development, deployment, and ongoing operations of a covered model or covered model derivative, including responsibility for compliance with obligations under this Part (5) Covered model.

(iii) mass casualties or at least $500,000,000 of damage resulting from an artificial intelligence model engaging in conduct that acts with limited human oversight, intervention or supervision and results in death, Page 2 DRS35250-MCa-139ASenate Bill 735-First Edition General Assembly Of North Carolina Session 2025 greatbodilyinjury,propertydamageorpropertyloss,andwould,ifcommitted by a human, constitute a crime specified in any general or special law that requires intent, recklessness or gross negligence, or the solicitation or aiding and abetting of such a crime;

or(ii) fine-tuningfine-tuningan anexistingcoveredmodel existing coveredmodel orcovered modelderivative usingaquantityofcomputingpowerandcostusingaquantityofcomputingpower andcost sufficienttoqualifyasacovered model.

– Modifying the capabilities of a covered model or covered model derivative by any means including, but not limited to, fine-tuning, providing such model with access to tools or data, removing DRS35250-MCa-139ASenate Bill 735-First Edition Page 3 General Assembly Of North Carolina Session 2025 safeguards against hazardous misuse or misbehavior of such model or combining such model with, or integrating such model into, other software.

– The SecretarySecretarymayconvene may convene an AI Innovation and SafetySafetyAdvisoryPanel Advisory Panel composed of representatives from industry, academia, civil liberties and consumer advocacy groups, and relevant state agencies.

Recommendations of this Panel shall be publiclypubliclyaccessible accessible and maymayinform inform future regulatory proposals.

(1) Implement reasonable administrative, technical and physical cybersecurity protections to prevent unauthorized access to, misuse of or unsafe post-training modifications of the covered model and all covered model derivatives controlled bybythe the developer that are appropriate in light oftheof the risks associated with the covered model, including from advanced persistent threats or other sophisticated actors.

(viii) describes in Page 4 DRS35250-MCa-139ASenate Bill 735-First Edition General Assembly Of North Carolina Session 2025 detail the conditions under which a developer would enact a full shutdown account for, as appropriate, the risk that a shutdown of the covered model, or particular covered model derivatives, may cause disruptions to critical infrastructure;

(4) Ensure that the safety and security protocol is implemented as written, including by designating senior personnel to be responsible for ensuring compliance bybyemployees employees and contractors working on a covered model or any covered model derivatives controlled by the developer, monitoring and reporting on implementation.

(5) Retain an unredacted copy of the safety and security protocol for not less than fiveyearsfive years afterthe coveredmodelis nolongermadeavailableforcommercial, public or foreseeably public use,, including records and dates of any updates or revisions.

(7) ConspicuouslyConspicuouslypublish publish a redacted copycopyof of the safety and securitysecurityprotocol protocol and transmit a copy of said redacted safety and security protocol to the attorney general;

DRS35250-MCa-139ASenate Bill 735-First Edition Page 5 General Assembly Of North Carolina Session 2025 (e) A developer shall not use a covered model or covered model derivative for a purpose not exclusively related to the training or reasonable evaluation of the covered model for compliance with State or federal law or make a covered model or a covered model derivative available for commercial, public or foreseeably public use if there is an unreasonable risk that the covered model or covered model derivative will cause or materially enable a critical harm.

(1) The investigator shall conduct investigations consistent with regulations issued bybythe the Secretary.

(i)adetailedassessmentofthedeveloper's stepsstepsto to complywith the requirements of this section;

andprovidedfurther,thatand theinvestigatorshallprovidedfurther,thattheinvestigatorshall not knowingly make a material misrepresentation in said report.

(h) A developer of a covered model shall annually, until such time that the covered model and any covered model derivatives controlled by the developer cease to be in or available for commercial or public use, submit to the attorney general a statement of compliance signed bybythe the developer's chief technology officer, or a more senior corporate officer, that shall specify or provide, at a minimum:

(i) an assessment of the nature and magnitude of critical harms that the covered model or covered model derivatives may reasonably cause or materially enable and the outcome of the assessment required bybythis this section;

(ii) an assessment of the risk that compliance with the safetysafetyand andsecurityprotocol securitymaybe protocol may be insufficient to prevent the covered model or covered model derivatives from causing or materially enabling critical harms;

and (iii) a description of the process used bybythe the signing officer to verifyverifycompliance compliance with the requirements of this section, including a description of the materials reviewed by the signing officer, a description of testing or other evaluation performed to support the statement and the contact information of any third parties relied upon to validate compliance.

Page 6 DRS35250-MCa-139ASenate Bill 735-First Edition General Assembly Of North Carolina Session 2025 (i) A developer of a covered model shall report each artificial intelligence safety incident affecting the covered model or any covered model derivatives controlled by the developer to the attorney general within 72 hours of the developer learning of the artificial intelligence safety incident or facts sufficient to establish a reasonable belief that an artificial intelligence safety incident has occurred.

provided, however, that this section shall not apply to a product or service to the extent that compliance would strictly conflict with thetermsofacontractbetweenafederalgovernmententityandthedeveloperofacoveredmodel.thetermsofacontractbetweenafederal governmententityandthedeveloperofacoveredmodel.

(n) Covered entities may voluntarily explore methods for disclosing to end-users when they are interacting with an artificial intelligence system, particularly where the nature of interaction is not immediatelyimmediatelyobvious. obvious.

Such entities maymayalso also explore labeling content generated by funded AI systems where there is potential for it to be mistaken for human-generated content.

(i) theidentitytheidentityoftheprospectivecustomer; oftheprospectivecustomer;

DRS35250-MCa-139ASenate Bill 735-First Edition Page 7 General Assembly Of North Carolina Session 2025 If a customer repeatedly utilizes computer resources that would be sufficient to train a covered model, the operator of the computer cluster shall validate said basic identifying information and assess whether such customer intends to utilize the computing cluster to train a covered model prior to each utilization.

(c) In complyingcomplyingwith with therequirements ofthis section,apersonthatsection,apersonthatoperatesacomputing operatesacomputing cluster may impose reasonable requirements on customers to prevent the collection or retention of personal information that the person operating such computing cluster would not otherwise collect or retain, including a requirement that a corporate customer submit corporate contact information rather than information that would identify a specific individual.

(1) Against a developer of a covered model or covered model derivative for a violation that causes death orbodilyharmto anotherhuman,harmto property, theft or misappropriation of property, or that constitutes an imminent risk or threat to public safety that occurs on or after January 1, 2026, a civil penalty in an amount not exceeding (i) for a first violation, five percent (5%) of the cost of the quantity of computing power used to train the covered model to be calculated using the average market prices of cloud compute at the time of training or (ii) for any subsequent violation, 15 percent (15%) of the cost of the quantityquantityof of computing power used to train the covered model as calculated herein.

(2) Against an investigator for a violation of this Part, including an investigator who intentionallyintentionallyor orwith withrecklessreckless disregard violates anyanyof of such investigator's responsibilities , or for a person that operates a computing cluster in violation of this Part, a civil penalty in an amount not exceeding (i) twenty-five thousand dollars ($25,000) for a first offense;

Page 8 DRS35250-MCa-139ASenate Bill 735-First Edition General Assembly Of North Carolina Session 2025 (3) Whether, in quality and implementation, the developer's safety and security protocol was comparable to those of developers of models trained using a comparable amount of compute resources.

(2) As a result of any such steps, the corporate structure of the developer or affiliated entities would frustrate recoveryrecoveryof of penalties, damages, or injunctive relief under this section.

(1) Prevent an employee from disclosing information to the attorney general or anyanyother other public body, includingincludingthrough through terms and conditions of employment or seeking to enforce terms and conditions of employment, if the employee has reasonable cause to believe the information indicates that (i) the developer is out of compliance with the requirements of this section or (ii) an artificial intelligence model, including a model that is not a covered model or a covered model derivative,poses anunreasonableriskofcausinganunreasonableriskofcausingormateriallyenabling ormateriallyenabling critical harm, even if the employer is not out of compliance with any State or federal law.

DRS35250-MCa-139ASenate Bill 735-First Edition Page 9 General Assembly Of North Carolina Session 2025 (c) An employee harmed by a violation of this section may petition the court for appropriate relief.

provided, however, that any information that is confidential, qualifies as a trade secret, or is determined bybythe the attorney general to likelylikelypose pose an unreasonable risk to public safety if disclosed shall be redacted from the complaint prior to disclosure.