Even if you have a fully unit tested app you will not be totally covered. Unit Test can only test units of code in isolation, but there is much more to an application than that: how units fit together -> Integration Tests, performance -> Performance Tests, how the user will use the app -> UI Tests + Manual Tests. Today we will take a quick look at what happened and why we had to implement a few Performance Tests to understand what was going on.
The client complained that the app was slow at loading a certain screen. We test the app as we develop it so we were pretty confused since it was working fine for us. I checked the response time of the server and the time for the app to become interactive, but it was really quick. No problems there. We only test on new devices since that’s what all developes have and have unit tests on top of it. So I tried testing on an older device, the idea being that since we were loading a very large amount of data it might have some problems with it. It was definitely slower.
At this point we started tinkering about performance tests. We could add them, but those would run on powerful mac hardware or our new devices so if the problem was like this one, which only happens when the device does not have enough hardware resources because of its age it would be useless.
After some debugging we found out that the problem was decoding the (very big) JSON response from the server was taking .5 seconds on an iPhone 11 and a whopping 4.5 seconds on a 5S (the oldest device we support).
We had this intuition from a while, but until we could actually test it and measure the real time taken in each method we could not say for use.
We could not drop support of older devices, but there was also not much we could do to increase the performance of the decoding process since it is just Apple’s frameworks. Migrating the whole app to a different framework that was faster just because of this would have required too big of a time commitment, but it would only be a band aid. It would not address the real problem, just improve the symptoms.
The real problem is the response from the server. There is no time when a text response should be even close to 2MB in size. Changing this would require relatively big changes for both the mobile apps and the rest api, but is the only real solution to the problem.
After changing the response to be paginated, as it should have been in the first place, the app now loads fine even on older devices. Once you have your fix in place make sure to test it again on the same device and similar network conditions. Always measure the results and do not simply trust your guts. In this case the difference was obvious from the naked eye, but that is not always the case.
Taking a few minutes to learn how to use Instruments, or the performance tool for your platform of choice, will pay back big time in these cases.