I've done the Live CD/USB, and that worked fine, but for the lack of being able to save data. Persistence is an option, but what I'm really looking for is a full fledged bootable install on a USB ...

Creating a bootable Ubuntu USB from a Windows system is a straightforward process that allows you to try out Ubuntu without installing it on your computer. This guide will walk you through the steps ...