Race in pickle.dumps/loads per-interpreter cache leaks references

[CharlieKerfoot]

Bug report

Bug description:

PyObject *dumps = state->pickle.dumps;
  if (dumps != NULL) {
      return dumps;
  }
  dumps = PyImport_ImportModuleAttrString("pickle", "dumps");
  if (dumps == NULL) return NULL;
  state->pickle.dumps = dumps;
  return dumps;

This is a check-then-act race. Two threads in the same interpreter can both run this code:

1. Thread A reads state->pickle.dumps → NULL
2. Thread B reads state->pickle.dumps → NULL
3. Thread A imports pickle.dumps, holds a strong reference R_A
4. Thread B imports pickle.dumps, holds a strong reference R_B
5. Thread A stores R_A into the slot
6. Thread B stores R_B into the slot, overwriting R_A

Now the slot owns R_B, and R_A is leaked forever.

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-149000: Fix race / leak in pickle XID per-interpreter cae #149002

[picnixz]

Do you have a reproducer for that? just for testing purposes.