This doesn't sound like the normal behavior. From the app description: "This indexing process may take some time, but it's a one-time event. Once this initial indexing process is complete, the app will store the index on your device, and any new photos you add to your photo library will be automatically added to the index on the subsequent app launches."
If your experience is different you may let the dev know so they can fix it.
Doing that I can see the loading bar for half a second when reopening the app. As stated on the app webpage this is certainly to update the database with any new images but it doesn't rebuild the entire database from scratch.
Maybe your cache gets deleted automatically when you close the app on your device. Make sure you have no other app that may do so.