How we solved our âPersistent Data Problemâ - Dev Blog
Made this post over on reddit today but figured weâd archive it here! If other devs would be interested in more posts like this about our game or gamedev in general let us know!
Even though Super Plexis is our first game, we committed ourselves to make something great from scratch without compromising the fun, Â the performance, or the original vision. But as Iâm sure many of you know, developing for mobile is not always conducive to lofty ambitions, and creating our gameâs back-end turned into a year and half long boss fight. Weâd like to share what weâve learned since it might benefit other indies (especially those with little-to-no budget) starting with: persistent data.
Hereâs two things we can probably all agree on:
1) Independent mobile games tend to have issues syncing account data between multiple devices.
2) Itâs frustrating when a mobile game that doesnât require multiplayer still requires an internet connection to play.
These two problems usually intertwine into what we call the âpersistent data problemâ.
So hereâs the cool thing about the account system we created for Super Plexis. We solved both of these problems without any monetary overhead or delayed user experience. We donât spend a dime on servers (if we could afford it we wouldnât be opposed to that), we donât make users sign into iCloud or anything like that, and users can play offline if they want. Hereâs the real kicker! All playersâ customizable profiles are publicly viewable in the ladder, and the system is very fast and memory efficient.
How did we do it? Iâll be speaking in terms of iOS because that was our launch platform, but rest assured, these tricks with GameCenter also have an equivalent in Google Play for Android devices (for which weâve recently started developing) . I wonât be talking too much about how we prevent cheating because that is a variable and game dependent subject. Here are just the basic ideas and feel free to ask any questions in the comments!
Pre-emptive TL;DR We found a way to use Game Center leaderboards as a server for our player data, bypassing the need for a paid server, and avoiding messy private databases like iCloud.
User requirements:
As long as the user is signed into Game Center, all of these features become available. A guest account is provided for users that donât want to be signed in.
The server:
Time to get comfy with binary arithmetic! We store entire accounts using hidden leaderboard score values / meta data. Each Game Center leaderboard score gives you 16 bytes of data (a 64 bit integer value named âscoreâ, and a 64 bit unsigned integer value named âcontextâ), and you can store 100 leaderboards if you want to. The trick is to string the leaderboard scores together. For example, in version 1.1.9 of our game, we are using 599 bits of data (75 bytes, 5 leaderboards) to store all of the ranked progress, unlockables (characters, portraits, skins, achievements, block sets), game-mode dependent  progress, important user preferences, stats, and account meta data. We scramble this data to prevent cheating and unscramble it when we are reading from it. In Game Center, leaderboard scores are usually âloadedâ and âreportedâ. For more information on that implementation, check out the GameKit API documentation regarding the GKScore object. In our game, we report and save the most recent score as opposed to âbestâ or âhighestâ score as that wouldnât make sense even for a poor manâs server like ours.
Offline accounts:
On each machine, we create one offline guest account and one offline player account for every Game Center user that opens the app. There are billions of ways to save / load account data on any machine. On iOS, âname.plistâ files are the easiest solution. We use a combination of encrypted plist files and serialized binary files. In general, itâs important to try to make these files unreadable so that you can minimize cheating. Anyway, whatever the user does on that device is always saved to their offline account. The localhostâs Game Center playerID looks like this: G:##########. In short, that is their sign-in username and password, so it is most convenient for the user to log them in immediately. If a change-of-users is detected, interrupt the game, immediately cancel any important background operations, save / sync the last userâs game state, and ask the new user to restart the app (or send them back to the title screen, this is also game dependent).
Online accounts reflecting offline accounts (longest and most important section): TL;DR, âGL HFâ
If the user is signed into Game Center and has a stable internet connection upon opening the app, we search for their scores in our leaderboards. In Super Plexis, this is when the first load screen says at the bottom âRetrieving data from Game CenterâŠâ. If this userâs score does not exist on a leaderboard, a new score is generated and submitted to that leaderboard. This just sets them up with some space in the online world. Now we have to use that space.
Recall âThe Serverâ, and lets use a small example. Suppose that we only had 16 unlockables in our game and they were all achievements. We only need 16 bits (2 bytes) to store the state of all these achievements in the leaderboards. It may look like this, Iâll separate my binary into byte sizes (pun very intended).
Achievements: 00000000 00000000 -> no achievements unlocked
Achievements: 00000000 00000100 -> achievement #3 unlocked
Achievements: 00000000 00001101 -> achievement #1, 3, and 4 unlocked
Achievements: 01010000 00001101 -> achievement #1, 3, 4, 13, and 15 unlocked
How to know which bit index reflects which achievement is up to your implementation. In Super Plexis, we created a parser object that links ambiguous bit indices with offline account object values. This way, when we set the offline value âSuper Cool Achievement Bro!â to âunlockedâ, the leaderboard bit that is linked to that achievement is set to â1â from â0â. There is one parser object per update, that way we can view player profiles in the ladder who have not yet updated to the most recent version (because your ânewerâ version remembers their âolderâ versions). This also allows us to rename data, move data, and resize data, without corrupting your progress when you update. As you can imagine, you cannot view the player profile of someone who has a more recent version of the game than you. Also we do not allow players of different versions to match with / invite each other in online multiplayer because we may have renamed, moved, added, or (hopefully never) removed a character.
For things like selectable characters, portraits, and block sets, you would normally store the related identifier string value in the server. We definitely do that for the offline accounts, but not the online reflection of them. As I said earlier, we have so little space to work with, so the data has to be very small. In this case, your best bet is to put all the character identifiers in a static array like this: [âmy_character_id_0â, âmy_character_id_1â, âŠ.]. For N updates, we require N of those arrays (one per version, again incase we rename, move, add, or remove one). Now we can just use the index of the identifier as the online data instead of using the identifier itself! So now, the offline account says your selected character is âmy_character_id_1â, but the bits that are linked to that account object value (the same bits that we are storing in the leaderboard to keep your data synced) say that your selected character is â1â. I.E, if you have 16 characters, you only need 4 bits to store which character the player has selected. When we save data to your offline account, we also try to report this compacted version of your account data to the Game Center leaderboards.
Syncing between multiple devices:
When the user is opening the app, we make sure that we are storing the identifierForVendor (returns from [UIDevice currentDevice].identifierForVendor) on the userâs leaderboard data. This is 16 bytes of data, so it takes up an entire leaderboard score. If the score does not exist, then we report a new one where those 16 bytes are set as the identifierForVendor. This is going to be used to check if the user has changed devices. Each time we report new data to the leaderboards, the current âidentifierForVendorâ is reported as well. This does not have to be saved as offline data. Also, we want to store some offline data like âlastAttemptedReportTimestampâ which will be updated and saved each time the user âtriesâ to report some data. The report can fail if the user is in offline mode, so we must track the last time this current device attempted to do that. Here are the three cases we need to look at:
1) If the user has not changed devices (if the offline identifierForVendor = the online identifierForVendor):
We do not need to sync at all. The best option is actually to report all of the userâs offline data to the leaderboards.
2) If the user did change devices, but the last successful report timestamp (the GKScoreâs date) is more recent than their offline dataâs âlastAttemptedReportTimestampâ:
Take all of the previously loaded leaderboard data and sync it to the offline account of the user. Save the offline account and report the current identifierForVendor so that we can record the next time a device change occurs.
3) If the user did change devices, but the last successful report timestamp (the GKScoreâs date) is older than their offline dataâs âlastAttemptedReportTimestampâ:
There is no way to know what we should do in this case. This usually never occurs but it is possible if the player is constantly switching devices and playing some of them in offline mode, then playing the same ones in online mode later. The trick here is to prompt the user and ask them to upload or download their data. If they donât know which one to choose, we offer a âplay offlineâ option where they can check their offline accountâs progress. If there is no missing progress, the user is guided to restart the app and tap âuploadâ, because the user would have been satisfied with their offline account. Otherwise, the user is guided to tap âdownloadâ, because the only way for there to be missing progress at this point is if their progress is in the leaderboards and NOT on their device.
Thatâs the gist of it! I left lots of small details out, and believe me there are many. This how we solved the persistent data problem in our game without paying for a server, using private databases like iCloud, or delaying the user experience. But when we are able to afford our own servers, we definitely will upgrade for all the obvious benefits (like cross platform play between iOS and Android devices, something that is not possible using this method). But for a dev team of only 2 guys, this method has been a life-saver on a budget. Hopefully it will help some of you as well!