Can't see the transaction/rollback code or query details, so I can't contribute any detailed insights.
Minimally, a deadlock occurs when 2 transactions must each complete before the other does. Naturally, the longer and more complex the transaction, the more likely deadlocks will occur.
For most circumstances, then, the shorter the transaction the better.
And for most deadlocks, the short solution is to ensure that the transaction is coded to retry when deadlock is detected.
If that doesn't resolve deadlocks in this transaction, you'll need to analyse it in detail. It can be torturous. Consider reading
https://dev.mysql.com/blog-archive/innodb-data-locking-part-3-deadlocks/ down to the word "retry", then follow the detailed cmds.