> the site can be down for 5 min max.
Then you had better get a new machine, and practice until you get it right.
A _lot_ has changed between 3.23 an 5.
PHP 4 -> PHP 5 will be another headache.
I would recommend using 5.1 or 5.5, so that you can wait another decade before the next update.
The data must be dumped, then reloaded. It is totally impractical to upgrade-in-place.
If you don't have the user's passwords, and you cannot get them converted, then you should probably run insecurely with old_passwords=1.
The conversion from latin1 to utf8 may or may not go easily. If you do it wrong, the text will be truncated. MySQL had not real concept of character sets until 4.1. See this for more discussion:
http://mysql.rjweb.org/doc.php/charcoll
There are hundreds of incompatibilities between 3.23 and 5.5. Most won't affect you, but a few could. Practice.
I hope you are not using ISAM tables. They are no longer supported.
CREATE TABLE ... TYPE=MyISAM -- the keyword "TYPE" has been replaced by "ENGINE".