For the longest time, TouchWiz was often criticized for its inability to get with the times and correct some of its longest-running points of criticism. Bloatware, a myriad of redundant features, outdated aesthetics and sub-par performance were the main things that stood out to many enthusiasts — particularly those enamored with stock Android. Through many Galaxy S and Note devices, TouchWiz has further iterated and improved, even attempting to re-brand itself under the name “Grace UX”. Despite such efforts, some of us still felt it provided an experience that was less than gracious, and the community at large opted to call the new Samsung Experience by its old name. But nothing stays the same for too long in this industry, and the Galaxy Note 8 has addressed one of Samsung’s oldest shortcomings: performance.
Whether this came from a natural evolution in specs or a targeted effort (likely both) is still up for debate — there’s no doubt that Samsung has extensively tweaked its software for this revision of the Note 8, and it’s also no secret that today’s hardware is measurably faster than last years. Indeed, on the hardware front the Galaxy Note 8 sports several key upgrades over last year’s Note7 … another bump in RAM to 6GB, an upgraded Snapdragon 835 processor (with the Adreno GPU clocked at 710MHz, unlike the S8’s 670MHz) and Samsung’s faster UFS 2.1 storage. All of this is important because, as we’ve said, Samsung’s improvements on the performance front have been tangible, but not relative to the leaps in fluidity and responsiveness that other devices have provided. This is why, when we looked at last year’s Galaxy Note 7, we couldn’t help but notice Samsung had yet again delivered what we called “embarrassing performance”.
With the Note 7, Samsung Still Delivers Embarrassing Real-World Performance
In last year’s article, we detailed some of the performance problems and annoyances we found in the Galaxy Note7 Snapdragon variants within our writer staff. XDA Contributor Eric Hulse provided various examples of reproducible lag that had plagued early adopters across a variety of usage scenarios — specific menus, scrolling in applications, and Samsung’s own typing experience through the stock keyboard. We provided several samples of these jarring stutters through gifs and videos, many of them showing the well-known “GPU profiling” bars — a histogram of the device’s last few frame times, which allow developers and enthusiasts alike to quantify their device’s smoothness, even if roughly, by allowing them to estimate the number of stutters and dropped frames. That article sparked a lot of debate within and outside of XDA, with multiple online blogs responding in retaliation to our claims or backing up our assessment and sharing their similar experiences.
The Galaxy Note 8 is a new phone, though, and a lot of things have changed since. TouchWiz has improved with its jump to Nougat, hardware has gotten better, and our standards have changed too. To be more specific, new devices have raised the bar for what we understand to be excellent performance — namely, the Pixel XL which keeps surprising all of us with its insane levels of fluidity (which we’ll demonstrate below). We thought it’d be appropriate to revisit the topic, and see how Samsung’s newest addition to the Galaxy lineup stacks up both against competitors and, most importantly, its notorious reputation. The Galaxy Note 8 offers perceptibly better performance with none of last year’s embarrassing stutters and jank, and we’ve been thoroughly surprised throughout our time with our units. Below we’ll document some of the improvements we’ve noted in the device’s performance and user experience, including comparisons with other devices.
Real World Smoothness
Methodology: In order to test real world fluidity, we won’t be presenting gifs or screenshots showing GPU profiling bars, but instead we will show you the extracted frame times plotted against other devices under the exact same usage scenario. We put together a tool to extract and parse the frame data, and a UI automation system that allowed us to build macros that mimic real-world use cases by simulating touch input — scrolling, loading new activities or windows, and compound tests with complex UI navigation. These tests were run across a Pixel XL, a OnePlus 5 (6GB) and the Galaxy Note 8, with all devices set to 1080p resolution and all devices fresh off a factory reset; this does mean that the Pixel XL is rendering less pixels than what most readers will be used to, potentially increasing performance by a noticeable margin. We made sure the tests were perfectly synchronized across devices, measuring the same actions at the same time, with multiple tests across each device to validate our results. Repeated tests continuously show minimal variance in the number of frames captured, though the number of total frames captured on each test varies significantly across devices. This is because these devices behave differently in their scrolling acceleration/final velocity, and set different baseline speeds for many actions and transitions (even at the same 1x setting).
The first thing we compared across devices was simple scrolling performance. It turns out that “dissecting” a scroll across devices does give us an idea as to what causes some of the performance differences we observe, especially across devices that largely feature the same hardware like the OnePlus 5 and Galaxy Note 8. Below you can both see the graphs of a simple three-scroll motion in the Google Play Store’s “Top Charts” list, pre-loaded ahead of the motion to ensure there’d be no disparities as the phones don’t fetch and load new elements from the internet, The motion is the same across the three devices, yet we see the Pixel XL showing absolutely no frames over the 16.6ms green line that indicates a missed frame. The OnePlus 5, by contrast, features more of those missed frames and they correspond to user input, similarly to what’s displayed on the Galaxy Note 8. There’s a reason for why these devices have higher frame times at the instants there’s user interaction, while the Pixel XL does not.
In the data above you can find these devices’ CPU frequency in their performance cores as the scrolling takes place. You can see how each of them reacts to user input (where frame times spike), with the Pixel XL quickly scaling up closer to its maximum frequency and remaining at a higher frequency throughout the motion. The OnePlus 5, by contrast, is a lot more erratic — its frequency steps are larger, and it very quickly jumps up and down. As a result, it’s likely that the OnePlus 5 does not encounter each new user interaction with the big cluster at an appropriate frequency, having to quickly ramp up to address the increase in CPU workload — by then, though, the phone might have already missed a few frames. The Note 8 sits somewhere in the middle of these two, with a median frequency sitting between the OnePlus 5’s lower recorded frequency and their top frequency of 2.36GHz (under this workload). The same disparities in frame times and CPU frequencies can be observed while scrolling through the main inbox within Gmail, again with a simple three-swipe scrolling scenario.
The OnePlus 5, once again, has the highest number of frame time spikes near the points at which our program simulates user input and scrolling. In this case, their “touch boost” succeeds at ramping the CPU frequency, but seemingly at meeting the entirety of the workload demand in time. The Pixel XL does see jank (which we’ll be calling frames or sets of frames above the 16.6ms green line) near the user input, but frame times progressively go down immediately after, and even then the spikes are much lower than what we see on both the OnePlus 5 and the Galaxy Note 8. Keep in mind that our perception of jank or stutters is not only about how many frames miss the 16.6ms target, but also how long a single or small group of frames takes to render. For example, if we saw a frame stuck for 700ms that would go from jank to freeze. While the Note 8 notably has fewer janky frames than the OnePlus 5 in nearly all of our tests, it is also capable of far higher frame times for individual frames, with repeated instances of the device blowing past our 50ms y-axis limit. Even then, none of these devices are “perceptibly” stuttery, with the results shown here matching the kind of scrolling performance most would deem entirely smooth.
So far we’ve seen that all of these devices are quite capable of smoothly handling simple scrolling motions, with the number of janky frames ranging from 0 to 13 percent. The Pixel XL is particularly impressive in this regard, and while it’ll continue its dominance across all of our tests, there’s an area where nearly all of these devices drop frames: loading new windows within an app. We’ve all seen, for example, some dropped frames when opening a Play Store listing and seeing all those beautiful animations putting everything in place. The following test includes a lot more element loading and UI navigation, being a composite UI test on the Google Play Store. You can click the toggle below to see the list of actions undertaken throughout the tests. The test is repeated three consecutive times, and results are displayed in the graphs below.
Play Store Test Actions
– Open application
– Open “Top Charts” list
– Scroll
– Open app listing
– Scroll
– Open “Read all reviews”
– Scroll
– Go back
– Go back
– Open side panel
– Open “Music”
– Scroll
– Go back
Nearly all of the devices we tested had tremendous spikes whenever they had to load a new window, be it a Play Store listing or a list of applications or songs. The scrolling portions are the ones that saw the smoother intervals by comparison. Loading the more complex windows (like a Play Store listing) can look perceptibly choppy in places, and I personally didn’t need any of these graphs to confirm that, but again the Note 8 performs much better than I expected. The Play Store isn’t a thoroughly lightweight application and the Note 8 does a good job at managing its transitions — at 12% jank, it had twice as many janky frames as the Pixel XL (relatively speaking, given total captured frames differ in number) and 33% less than the OnePlus 5 even though both had the same average frame time throughout the test. Mind you, this doesn’t mean that the OnePlus 5 effectively runs at 49 frames per second — frames are only recorded when the screen updates, and the only points at which there is continuous motion that add 60 frames every second to our plot is when there is continuous scrolling, or long loading instances.
Another popular application we decided to look at is YouTube, with the steps detailed under the toggle below. Once more, the test is repeated three times, and keep in mind that video playback within the app is not represented in the frame plot in any way, as none of the video frames or frame times are recorded.
YouTube Test Actions
– Open YouTube
– Search for “Nyan Cat”
– Open video
– Minimize video
– Scroll
– Search for “xda-developers”
– Open xda-developers channel
– Go through videos
– Scroll
– Go through playlists
– Go through channels
– Go back to playlists
– Go back to videos
– Go back to channel home
– Go back to search screen
– Go back to YouTube homescreen
– Swipe video away
Yet once again, we see the Galaxy Note 8 having a higher jank percentage than the Pixel XL, but less than the OnePlus 5, with the OnePlus 5 having the highest average frame time. This isn’t the most intensive test, but YouTube is a very popular application and one that I personally use for hours every day on any given phone. I was very pleased with the Galaxy Note 8’s in-app performance in all of these compound tests (and IRL), and while performance isn’t quite on the level of the Pixel XL, it’s quite literally better than what I’ve been used to as far as smoothness specifically is concerned, and much much better than the stuttery mess I encountered in many instances with my Galaxy Note 7 units (both the Snapdragon and the Exynos variant). This test has plenty of navigation, horizontal swiping and thumbnail loading, so I was pleased to see that these devices performed rather well, and that the Galaxy Note 8 did such a good job.
Finally, the last composite test I ran was through the Gmail application, three times on the same Google account under the same network.
Gmail Test Actions
– Open Gmail
– Scroll twice
– Open Side Panel
– Open “Spam”
– Scroll down
– Scroll up
– Open spam email
– Go back
– Go back
– Compose email
– Go back
– Open side panel
– Scroll side panel
– Open Settings
– Open General Settings
– Go back
– Go back
There’s not much to do in Gmail other than scrolling through lists, opening emails and composing new messages — that’s pretty much what this test replicates, as it navigates through various Gmail interfaces, loading various activities with long and diverse animations. This is probably why it’s one of the worst performing tests, as it only includes a small section of scrolling near the beginning. As we saw in previous tests, scrolling through a Gmail list also proved bumpier than the Play Store app list (which surprised me) on these devices and with these swipes. In the end and like clockwork, we see the same order: Pixel XL is first, Galaxy Note 8 janks a tad more, and the OnePlus 5 further a bit more than the Note 8. All in all, this last test reinforces the general induction I arrived to after spending a week careful obsessing over the Galaxy Note 8: in-app performance is now really nothing to complain about, and a far cry from the old TouchWiz we trashed so vigorously over the years.
Sadly, in-app performance isn’t the entire story (though it’s a huge part of what we look for in a phone). The unfortunate reality is that the Galaxy Note 8 still features a number of micro-stutters outside of third-party applications, many of which were more difficult to capture with our tool. For example, take the graphs above which show opening and closing (swiping) the app drawer in the homescreen, multiple times (TouchWiz Launcher, OxygenOS launcher, Pixel Launcher). The Galaxy Note 8 will almost always drop frames whenever it’s summoned by swiping up, and the first jank upon any new start of the launcher is always dramatically bad, shooting way past the 50ms limit in our graph and being a lot more jarring and noticeable that subsequent swipes. Other areas of the user interface have shown similar stutters, including the occasional stutter when unlocking the phone and the dropped frames when accessing the homescreen’s left-most panel, which now houses Bixby. It’s been several years since Samsung’s left-most homescreen has had spectacularly choppy transitions, and while it’s better than the Flipboard mess of previous years, it now manages to embarrass Samsung’s own first-party assistant service. But of course, this is just a launcher — swap it for something better, and the problem’s done with in this instance. The few stutters that remain in system animations when transitioning across applications aren’t much of a problem at this stage, and in my experience they are so minimal compares to previous years that I don’t think many people will be bothered by them.
App launching times & Multi-tasking
Methodology: We measured cold-start launch-time performance of Gmail, the Play Store and YouTube on the Pixel XL, OnePlus 5 and Galaxy Note 8. Keep in mind that we are not measuring the time it takes for an app to be fully rendered with all its elements drawn on screen. Rather, we are using a proxy by recording the time it takes for the app to create the main activity of the application. The time measure we include encompasses launching the application process, initializing its objects, creating and initializing the activity, inflating the activity’s layout and drawing the application for the first time. It ignores inline processes that do not prevent the initial display of the application, which in turn means the recorded time is not affected by extraneous variables such as network speed fetching burdensome assets. Also keep in mind that the phones tested immediately top up their CPU frequencies whenever an application is launched, minimizing CPU bottlenecks. We cycled through the three applications, and opened each of them 150 times, to look at how these phones’ app-launching capabilities perform over time. This throttling is unconventional and we’ve pushed the phone way past the limits you would encounter in real-world use-case scenarios, but even then, only one device was afflicted by severe performance degradation.
In this case, the OnePlus 5 takes the crown, featuring the fastest app opening speeds — even if these numbers don’t include finalizing all assets, it’s still a very solid predictor that closely matches our perceived real-world experience. The OnePlus 5 is a champion at opening apps with incredible stability even as temperatures rise throughout this 15 minute test of intense app-opening, which keeps the CPU at its higher frequencies for an unreasonable amount of time. While the OnePlus 5’s performance cluster frequency doesn’t climb past 2.36GHz in most usage scenarios (including the tests shown above), it does shoot to 2.5GHz (and thus higher than the Note 8’s 2.36GHz) when opening applications. The Galaxy Note 8 is the subject of this analysis, though, and it performs almost as well as the OnePlus 5, with slightly higher opening times in every application and the same overall pattern, with near-identical performance gaps for Gmail and the Play Store and almost the same gap between those two and the YouTube app. We included the Pixel XL given we had included it in every other test, though it’s no match for the Note 8 and OnePlus 5. Its slower storage (a factor in this instance) and now-outdated Snapdragon 821 were unable to achieve the same opening times, with significantly worse throttling throughout the test.
But app opening speeds do not tell us the full story, there’s another key aspect of an excellent Android experience and that is multi-tasking. While we attempted to measure app opening speeds from cold-starts, under most conditions we see what we call warm (and luke-warm) starts. And having an additional 2GB of RAM goes a long way in making the Galaxy Note 8 a much better performer when it comes to juggling applications, with this increase being one of the largest functional bumps in RAM Samsung has brought forth. Many videos and hands-on speed tests or comparisons have already shown the Galaxy Note 8 to have much better app-holding capabilities than the 4GB Galaxy S8 and S8+, and while we don’t always agree with such tests we echo the same opinion — the Note 8 is consistently better at holding up to twice as many applications as the Galaxy Note 7, and even the S8 when we factor in bigger games or heavier applications. I’ve personally been very critical of Samsung’s RAM management in the past, and while it’s obvious that a 6GB device would not (and really should not) have such issues in today’s app environment, I am still very happy to see the problem addressed (in one way or another). The Note 8 hasn’t disappointed me with its ability to keep applications in memory, and that really helps enhance the overall real-world performance of this device.
Note 8 Real-World Performance & Final Thoughts
As my XDA bio has stated for the past two years, I am very obsessive when it comes to performance (and button tactile feedback, but that’s a story for another day). At the same time, long-time readers might be aware that I’ve also been a fan of the Galaxy Note line-up ever since the Galaxy Note 3 won me over — I have owned every Note device since, in some cases multiple variants of each. But despite the fact that I kept coming back to the Note line, I was always aware that I was sacrificing an extremely important aspect of the user experience which I sought to fix through modifications, custom ROMs and kernels, and a lean and laid back approach to applications. Having to restrain my user experience in what’s otherwise the quintessential power usage device is a contradiction that never escaped me, yet it was something I had to live with device after device in order to make use of the valuable S-Pen and excellent multi-tasking capabilities (which other devices severely lacked back then).
When we wrote our assessment of the Galaxy Note 7’s real-world performance, we faced a salvo of criticism (much of it being the result of everlasting fanboy wars). This time around, we sought to show you our findings through a data-driven approach that attempts to capture and quantify how these devices perform on equal footing, under the exact same workload, with the same starting parameters. We’ve validated these results and have run these tests multiple times, with minimal variance (the Pixel XL, in particular, seems to operate in a perfect Newtonian clockwork universe here). But as confident as we are about the Galaxy Note 8’s real-world performance today, we’ve only provided a snapshot in time against a specific set of devices, and Samsung phones in particular are notorious for “slowing down” over weeks and months. Most reviewers, for example, initially sing praise towards Samsung for finally fixing TouchWiz’s performance woes, only to begin walking back their evaluations weeks later, or in re-reviews of Samsung devices. This has happened time and time again, and while we never were particularly fond of Samsung’s real-world performance out of the box, we certainly can’t guarantee the Galaxy Note 8 won’t regress with age either.
Yet at the same time, we can’t help but be surprised that the Note 8, so many years later, manages to minimize one of the series’ most criticized aspects. We were really critical of Samsung and the Note 7 last year, but so far there is no indication that the Galaxy Note 8 suffers from the same lockups, jarring animations, reproducible stutters and general lack of polish that left us disappointed in 2016. Past the examples provided in this article, I have personally had no complaints with my day to day usage of the device, and I have only noticed one small keyboard lockup whilst updating applications, which arguably excuses it. Other than that, though, it has been extremely serviceable with fast and fluid in-app performance, much better responsiveness than last year, and night-and-day improvements in app-holding capabilities. All of this allows the Note 8 to shine brighter as a power user phone than its predecessors, perhaps not in relative terms (given phones in general have gotten much better) but it is certainly everything I wished previous Note devices were in this regard. But do not be mistaken — it is still not perfect. It is still below the bar put forth by devices like the Pixel XL, and its biggest weak points remain precisely in those areas of the phone where Samsung is most involved, namely the system UI and some stock applications. It’s also nowhere near as snappy as the OnePlus 5, which combines an aggressive approach to performance with zippy animations to deliver an extremely responsive UX. It’s much harder for us to accurately measure the time it takes for actions within applications to be carried out, but I am confident that the OnePlus 5 would be a clear winner in this area.
That being said, I am very happy with the Note 8 so far. It is a compelling powerhouse of a phone, and while it might have priced itself a bit north of what I would have hoped, it does offer tremendous value in a market that still hesitates to challenge Samsung in the niche it has held for so many years — productivity devices with stylus support. My only complaint is battery life, which has been the worst I’ve had in any Note device to memory. This is puzzling to me, given I don’t feel a substantial improvement (if any) over the Note 5, which had a smaller 3,000mAh battery, an older processor and a default resolution of 1440p for its display settings. If it wasn’t for fast charging, as well as fast wireless charging, this would be unacceptable… and I often wonder whether we traded away some of that endurance for either more features most don’t need, or the performance this very article praises. However, that is a topic for another day. As it stands, the Galaxy Note 8 has proved to us that Samsung no longer delivers embarrassing real-world performance. I’m quite satisfied with the phone’s performance out of the box, and I really hope it can last this way — we’ll be sure to let you know if that’s not the case, but for now, I can say the Note 8 hasn’t disappointed us.
What do you think of newer Galaxy devices and the modern TouchWiz/Samsung Experience? Sound off in the comments!
0 comments:
Post a Comment