I am seeking an answer that will allow persistence without needing an SQL server to become placed on the host and may be bundled up together with a desktop application. Does this type of factor exist?

You could utilize an embeddable database designed in Java like Java DB (Sun's supported distribution of Apache Derby), HSQLDB, or H2 (through the creator of HSQLDB). H2 has my preference performance smart featuring smart (some evaluations here, here and here) and due to the compatibility mode along with other databases (excellent when H2 isn't your target database but maybe irrelevant inside your context).

I believe you would like something similar to SQLite (Java bindings) (a quick flat-file general purpose embeddable database).

You may take a look at SQL Server Compact Edition. It's most of the same features because the full version of SQL Server, is free of charge and doesn't need a separate installation.