Yeah, the non-portable build just stick to the set Windows standards about where to store files, so they may appear at different places depending on the OS version.
In the game folder there's a "Saira.ini" file in which you can modify to enable or disable the portable mode, if the value is set to 1 the game will write the save data to it's own applicaion folder. The reason for the two different modes is that one is designed to work with all the Windows standards, where savegames are tied to user accounts and DEP doesn't warn about the game. The portable version can not work as an installer since the new versions of Windows doesn't allow programs to write to protected paths such as Program Files.
Don't get me wrong, this is AWESOME. Or I'd rather say, Mac OS and Linux have worked like this since ages ago, which is why those systems have been more or less free from viruses, so it's about time that Microsoft finally introduced this (10 years later than everyone else) so their operating systems is no longer completely exposed from attacks from malicious software, and in the future I will build my games to offer both a portable and non-portable versions so that everyone use what they prefer.