Can I turn off RFI numbering by project stage in Procore once it is on?

No. Enabling Prefix RFI Numbers by Project Stage is irreversible, and Procore shows a confirmation dialog for that reason.

The setting lives on the project level RFI settings page, alongside default turnaround, overdue reminders, response visibility and the email notification matrix. Everything else on that page can be changed back. This one cannot.

Decide it before the first RFI is raised, because the prefix changes how every number reads and how anybody searching for one will find it. If you are unsure, leave it off: an unprefixed scheme can be lived with, and a prefixed one cannot be undone.