Instead of being global to all playlists, these two modes "repeat" and "random" could be per playlist settings.
This way, without further tweaking you could, for example, create and save a playlist for all Pink Floyd albums played in order which could repeat when all albums have already been played once, but switching to a more general playlist, this one would play random tracks from whaver artists the playlist contains.