No, I don't think it will be too hard. Well, I haven't managed to defeat it but just because I wanted to give feedback quickly, so I didn't spend time grabbing enough gold from the first two kingdoms. From then on I just put a goblin inside to check the links in the other kingdoms and be defeated.
For the moment I haven't upgraded the tower yet but I think I might have to sooner or later lol. The only thing I didn't understand was if the special floors power up specific minions or if they do more damage to specific enemies.