Django get_or_create IntegrityError due to racing condition

In Django, get_or_create is not a thread safe method. So, it might results in racing condition. There is no standard solution available. More info on this issue in https://code.djangoproject.com/ticket/13906

Possible solutions from stackoverflow answer, and also better explanation about the problem

Basically get_or_create can fail – if you take a look at its source,
the you’ll see that it’s: get, if-problem: save+some_trickery,
if-still-problem: get again, if-still-problem: surrender and raise.

This means that if there are two simultaneous threads (or processes)
running create_or_update_myobj, both trying to get_or_create the same
object, then:

first thread tries to get it – but it doesn’t yet exists, so, the
thread tries to create it, but before the object is created…
…second thread tries to get it – and this obviously fails now,
because of the default AUTOCOMMIT=OFF for MySQLdb database connection,
and REPEATABLE READ serializable level, both threads have frozen their
views of MyObj table. subsequently, first thread creates its object
and returns it gracefully, but… …second thread cannot create
anything as it would violate unique constraint what’s funny,
subsequent get on the second thread doesn’t see the object created in
the first thread, due to the frozen view of MyObj table So, if you
want to safely get_or_create anything, try something like this:

 @transaction.commit_on_success
 def my_get_or_create(...):
     try:
         obj = MyObj.objects.create(...)
     except IntegrityError:
         transaction.commit()
         obj = MyObj.objects.get(...)
     return obj

Transaction.commit_on_success helps to get rid of frozen view of MyObj by creating new transaction.

This entry was posted in Django. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *