There was a time when general consumers would buy their smartphones by going into smartphone retailer shops and relying on the expert knowledge of the salesmen of the store and bet on only available legacy brands like Nokia, LG, Samsung, Microsoft Lumia, HTC & Apple, and later on, maybe Motorola And the job of those smartphones were to have a VGA or 2MP camera, only one camera precisely, one touch screen and able to browse net at Google Chrome and Facebook & Orkut and some hardcore gamers having 512MB to 1-1.5GB RAM with games like  Fruit Ninja, Sudoku into gradually My Talking Tom, Hill Climb Racing, Angry Birds, Temple Run, etc.

Standing in today’s time of 4-5 cameras at back, 2 at the front, having curved (foldable for Rich for now) 6.7inch display, 8k video recording, 16GB RAM, and need to play the tagline “Console Level Graphics” game from PUBG, PES, Asphalt, Fortnite ( Not in the Antitrust Controversy of Apple & Epic; that’s for another day WE are spoiled consumers and knowledgeable too. We are skeptics, we research and invest hours before deciding on one brand, and then on the model, we also need to know in-depth about camera sensors and processors-sub processors-AI/neural engines-ISPs and even security chips. We will watch our favorite tech YouTuber’s reviews of the newest and freshest chocolates of Apples, Samsungs, and their Davids: Chinese counterparts ( autocorrected into counterfeits, the wrong statement, no worries I myself use one Chinese brand smartphone) Xiaomi, Vivo, Oppo, Oneplus and Huawei. Pixels, Nokia, Sony Xperia, Asus, and Motos are not mainstream buying habits in India so if you use those don’t hold me for the offense of not taking the name of your smartphone brand in the same line.

In this stark contrast between using smartphones in 2012 and now in 2022, what’s the problem that let me write an entire article? Yes, there are! 

Today’s smartphone users are much more tech-savvy, for example, take a look at home, my mother when bought her first smartphone in 2012 she bought HiTech, one forgotten Chinese brand which the salesman pushed her into, then in 2015 she bought LAVA one family member suggested her into, recently she bought Samsung M series, and guess what? No, she just asked me to confirm her choice. She did not go for Samsung brand image, she watched a lot of reviews, comparisons, camera tests, battery tests, service experiences, and performance tests to reach a decision.

And My mother is not the only one. You all might experience it, right?

Here the problem lies... No, not with ourselves, We have been much smarter in choices, but the smartphone companies became much smarter in manipulating our choices also.

People who see a lot of youtube content of smartphone reviews, you will see reviewers ( can refer to MKBHD, Unbox Therapy, Mr. Mobile, MrWhosTheBoss, C4ETech, Technical Guruji, UrAvgConsumer, Linus Tech Tips, GeekyRanjit, etc) come performance test or reviews or performance comparison between different smartphone CPU, GPUs of different companies talk about Real World Performance or Like say, “ We all know benchmarks don’t always translate into real day to day work, etc” but also shows the AnTuTu, 3DMark, Geekbench, AI, PCMark for Android Benchmarks of those smartphones. 

And the PROBLEM IS SMARTPHONE COMPANIES MANIPULATE THESE BENCHMARKS OR CHEAT IN THESE BENCHMARKS: Practically smartphone companies get caught cheating with smartphone performances!

It is happening from the time when the smartphone performance benchmarks come and no consequences of these actions, Funnily strange enough?

What are the Benchmarks and what it is supposed to do?

A benchmark, at its most basic level, is an app that determines how quickly your phone can perform certain tasks. It puts the phone through a series of tests to determine its maximum capacity. The theory is that if you put enough stress on the phone, it will function at its best. That performance may be graded numerically, with each number positioned in relation to the others.

This is the problem with benchmarks. There is no such thing as an absolute scale. Every benchmark has its own scale for grading applications. You can't compare two benchmark programs; you can only compare two devices that are both running the same benchmark.

And what it is supposed to do and how?

Benchmarking each smartphone literally takes hours.  Benchmark testers basically begin the test, wait till it finishes evaluating and record the result. It’s just not done yet. You start all over again and perform the same operation a couple of times. Mind you this is only for one benchmarking test. The reason behind performing the test in multiple iterations is that the result varies each time. So, why does that happen? The working of the smartphone CPU is quite complex. At times it might use just two cores to do a job, while the same job might require four cores if there are multiple applications in the background. So, the evaluated performance is not even all the time. This is why there is a need to run multiple iterations and later average them out.

It’s always a wise thing to take these benchmarking numbers with a grain of salt and there is a multitude of reasons why you should do that. In any field that involves electronics, there is tolerance. So, the performance is not always the same. Having said that – the consistency of all the electronic components has greatly improved.

For example, the compiler used for evaluating ARM-based silicon is not the same for gauging the performance of an Intel-based chipset. Now if you want to take a look at some of the most common Benchmark apps, there are

AnTuTu Benchmarking 

Geekbench 3 Benchmarking

Quadrant Benchmarking

Basemark Benchmarking

GFX Bench

Benchmark cheating & background manipulation :

The action of whitelisting a benchmark app by optimizing the maximum performance of the smartphone just for specifically targeted benchmark app is generally considered by the industry as cheating since it defeats the purpose of a benchmark, which is to reflect user experience for day to day use.

Back in 2013 When Anandtech learned about some of Samsung's antics with the GPU of Exynos chipsets on the Galaxy S4, it blew up into a larger investigation of the practice among many of the mobile manufacturers at the time - with all of them being found guilty. The Samsung case finally resulted in a winning $13.4 million class-action lawsuit verdict against the business — AnandTech, a tech analytics website, was the first to notice the discrepancy and was even included in the court filing.

Over the years, the naming and shaming worked, as merchants rapidly abandoned such tactics for fear of media reaction - the drawbacks significantly surpassed the rewards.

In 2017, Apple was also caught in this manipulative practice, no they did not cheat on benchmarks, Apple has been fined 25 million euros (£21 million, $27 million) for intentionally slowing down older iPhone models without informing customers.

The DGCCRF, France's competition, and fraud watchdog levied the penalties, claiming that customers were not informed. Apple confirmed that it did slow down some iPhones, but said it only did so to "prolong the life" of the devices. the company confirmed it did slow down some models as they age, but not to encourage people to upgrade.

It claimed that as the lithium-ion batteries in the gadgets aged, they became less capable of meeting peak current demands. As a result, an iPhone may shut down abruptly to safeguard its electronic components.

After that iPhone 6, iPhone 6s, and iPhone SE received a software upgrade that "smoothed out" battery performance. A client verified the practice after posting performance tests on Reddit, claiming that their iPhone 6S had slowed significantly as it aged, but had suddenly speeded up again once the battery had been replaced.

But every OEM did not get the same clean chit from users, OEMs of all sizes (including Samsung, HTC, Sony, and LG) took part in this arms race of attempting to fool users without getting caught. 

One investigation on the OnePlus 3T caught that OnePlus was targeting these benchmarks by name, and was entering an alternate CPU scaling mode to pump up their benchmark scores.

Oneplus at that time assured they will not again target any benchmark and will not use this coded manipulation in cheating. But without any such regulations and legal implications. But again in the case of Oneplus 5, they went one step further, the cheating mechanism is blatant and aimed at maximizing performance, unlike last time which did not increase scores by much on average, but did reduce variance and thermal throttling.

. XDA Developers literally put the headlines in their posts, “ [UPDATED STATEMENT] Do NOT Trust OnePlus 5 Benchmarks in Reviews – How our Review Unit is Grossly Cheating at Benchmarks”.

And ..Again… With the Oneplus 9, they cheated in the benchmark again! Anandtech in their test discovered weird behavior on the OnePlus 9 Pro flagship: Popular apps are performance limited, while benchmarks are unaffected. After repeated incidents, year after year Performance benchmarking platform Geekbench, at last, delisted the OnePlus 9 and the OnePlus 9 Pro.  Geekbench officially stated, “ It's disappointing to see OnePlus handsets making performance decisions based on application identifiers rather than application behavior. We view this as a form of benchmark manipulation. We've delisted the OnePlus 9 and the OnePlus 9 Pro from our Android Benchmark chart.”

In 2018 Huawei P20, Huawei Nova 3, and Honor Play were delisted from 3DMark for benchmark cheating.

OPPO Find X and F7 were delisted from 3DMark for benchmark cheating in that year only,

But the big fish caught in 2020, when the SOC maker Mediatek get caught cheating and yes, these chips were in most of all Chinese OEM smartphones, MediaTek was cheating on benchmarks by creating a whitelist of applications that would run the device in a special ‘Sports Mode’ that significantly boosts the performance of a device beyond it’s real-world day to day capabilities and would be unsustainable in day to day usage. Shockingly the cheating mechanism had been hiding in plain sight for years in some innocuous-looking code. Hidden in the firmware there is a .xml file with a list of popular applications with various power management tweaks, including a list of popular benchmarks that would kick off MediaTek's Sports Mode.

 After the article came out about Mediatek malpractice, first of all, they denied the allegation directly. Their official press release says, ‘We believe that showcasing the full capabilities of a chipset in benchmarking tests is in line with the practices of other companies and gives consumers an accurate picture of device performance’’

Other SoC vendors and smartphone vendors that got caught “cheating” in the past have owned up to their behavior and cleaned up their acts.  MediaTek doesn't seem to back down from this at all. In fact, the company published a blog titled ‘Why MediaTek Stands Behind Our Benchmarking Practices’ on their website shortly after the Anandtech article on catching Mediatek malpractice went live.

And all comes down to this 2022, February-March Samsung Mobile.

the company is accused of throttling 10,000 Android apps—but not benchmark apps. It sounds like the scheme OnePlus was caught running last year. Instead of boosting the SoC speeds when a benchmark app is running, Android OEMs are now turning down phone performance any time a benchmark app isn't running. It's like benchmark cheating but in reverse.

Samsung's throttling app is called the "Game Optimising Service." 

Users of the Korean message board Clen.net found wildly different benchmark scores depending on whether benchmark apps had their original names or not.

The list also includes every popular third-party app you can think of—Netflix, Disney+, TikTok, Facebook, Twitter, Amazon, among others. The only apps you reliably won't find on this list are benchmarking apps. Geekbench, 3D Mark, PCMark, GFXBench, Antutu, CPDT, and Androbench are all missing from the list.

And the result was not so much as easy criticism, Geekbench accuses Samsung of benchmark manipulation, delisting the last four years of Galaxy flagships.

Regulatory Demand : 

Unfortunately, these smartphone companies(OEMs) and SOC makers are not bound to not do this malpractice or cheating by any law not in India nor in any specific law. Benchmark cheating, Performance throttling, and performance push in certain actions and apps are not defined by the state or any common stakeholder bodies. We can say these make companies look bad in the eyes of consumers and reviewers, but this reputation loss can be the right redressal method. These malpractices by OEMs and smartphone chip makers are clear manipulation to trick consumers to buy into their products-services which should come under the Consumer Protection Act,2019 in India and also by whitelisting benchmark apps or some specific apps are an abuse of power and position and raises antitrust concern for rest of the apps and even competitive OEMs it should be taken into consideration By Competition Commission of India under Competitions Act. There is a massive regulatory void in regulating and controlling these under the hood CPU-GPU level cheating or antitrust actions by these companies for over 10 Years. It is now time for the stakeholders and government should take note of this situation more seriously and actively, GeekBench or 3D Mark Delist can be the investigation starter but there should be proper procedural safeguards for consumers and competitors and a proper checking process and regulating structure with legal statutory backing. I hope my short article can be the start to the end of mischief by these smartphone companies ( OEMs & SOC makers)

BTS: Thank You AnandTech & XDA Developers for being the Angels in Demon.

There was a time when general consumers would buy their smartphones by going into smartphone retailer shops and relying on the expert knowledge of the salesmen of the store and bet on only available legacy brands like Nokia, LG, Samsung, Microsoft Lumia, HTC & Apple, and later on, maybe Motorola And the job of those smartphones were to have a VGA or 2MP camera, only one camera precisely, one touch screen and able to browse net at Google Chrome and Facebook & Orkut and some hardcore gamers having 512MB to 1-1.5GB RAM with games like  Fruit Ninja, Sudoku into gradually My Talking Tom, Hill Climb Racing, Angry Birds, Temple Run, etc.

Standing in today’s time of 4-5 cameras at back, 2 at the front, having curved (foldable for Rich for now) 6.7inch display, 8k video recording, 16GB RAM, and need to play the tagline “Console Level Graphics” game from PUBG, PES, Asphalt, Fortnite ( Not in the Antitrust Controversy of Apple & Epic; that’s for another day WE are spoiled consumers and knowledgeable too. We are skeptics, we research and invest hours before deciding on one brand, and then on the model, we also need to know in-depth about camera sensors and processors-sub processors-AI/neural engines-ISPs and even security chips. We will watch our favorite tech YouTuber’s reviews of the newest and freshest chocolates of Apples, Samsungs, and their Davids: Chinese counterparts ( autocorrected into counterfeits, the wrong statement, no worries I myself use one Chinese brand smartphone) Xiaomi, Vivo, Oppo, Oneplus and Huawei. Pixels, Nokia, Sony Xperia, Asus, and Motos are not mainstream buying habits in India so if you use those don’t hold me for the offense of not taking the name of your smartphone brand in the same line.

In this stark contrast between using smartphones in 2012 and now in 2022, what’s the problem that let me write an entire article? Yes, there are! 

Today’s smartphone users are much more tech-savvy, for example, take a look at home, my mother when bought her first smartphone in 2012 she bought HiTech, one forgotten Chinese brand which the salesman pushed her into, then in 2015 she bought LAVA one family member suggested her into, recently she bought Samsung M series, and guess what? No, she just asked me to confirm her choice. She did not go for Samsung brand image, she watched a lot of reviews, comparisons, camera tests, battery tests, service experiences, and performance tests to reach a decision.

And My mother is not the only one. You all might experience it, right?

Here the problem lies... No, not with ourselves, We have been much smarter in choices, but the smartphone companies became much smarter in manipulating our choices also.

People who see a lot of youtube content of smartphone reviews, you will see reviewers ( can refer to MKBHD, Unbox Therapy, Mr. Mobile, MrWhosTheBoss, C4ETech, Technical Guruji, UrAvgConsumer, Linus Tech Tips, GeekyRanjit, etc) come performance test or reviews or performance comparison between different smartphone CPU, GPUs of different companies talk about Real World Performance or Like say, “ We all know benchmarks don’t always translate into real day to day work, etc” but also shows the AnTuTu, 3DMark, Geekbench, AI, PCMark for Android Benchmarks of those smartphones. 

And the PROBLEM IS SMARTPHONE COMPANIES MANIPULATE THESE BENCHMARKS OR CHEAT IN THESE BENCHMARKS: Practically smartphone companies get caught cheating with smartphone performances!

It is happening from the time when the smartphone performance benchmarks come and no consequences of these actions, Funnily strange enough?

What are the Benchmarks and what it is supposed to do?

A benchmark, at its most basic level, is an app that determines how quickly your phone can perform certain tasks. It puts the phone through a series of tests to determine its maximum capacity. The theory is that if you put enough stress on the phone, it will function at its best. That performance may be graded numerically, with each number positioned in relation to the others.

This is the problem with benchmarks. There is no such thing as an absolute scale. Every benchmark has its own scale for grading applications. You can't compare two benchmark programs; you can only compare two devices that are both running the same benchmark.

And what it is supposed to do and how?

Benchmarking each smartphone literally takes hours.  Benchmark testers basically begin the test, wait till it finishes evaluating and record the result. It’s just not done yet. You start all over again and perform the same operation a couple of times. Mind you this is only for one benchmarking test. The reason behind performing the test in multiple iterations is that the result varies each time. So, why does that happen? The working of the smartphone CPU is quite complex. At times it might use just two cores to do a job, while the same job might require four cores if there are multiple applications in the background. So, the evaluated performance is not even all the time. This is why there is a need to run multiple iterations and later average them out.

It’s always a wise thing to take these benchmarking numbers with a grain of salt and there is a multitude of reasons why you should do that. In any field that involves electronics, there is tolerance. So, the performance is not always the same. Having said that – the consistency of all the electronic components has greatly improved.

For example, the compiler used for evaluating ARM-based silicon is not the same for gauging the performance of an Intel-based chipset. Now if you want to take a look at some of the most common Benchmark apps, there are

AnTuTu Benchmarking 

Geekbench 3 Benchmarking

Quadrant Benchmarking

Basemark Benchmarking

GFX Bench

Benchmark cheating & background manipulation :

The action of whitelisting a benchmark app by optimizing the maximum performance of the smartphone just for specifically targeted benchmark app is generally considered by the industry as cheating since it defeats the purpose of a benchmark, which is to reflect user experience for day to day use.

Back in 2013 When Anandtech learned about some of Samsung's antics with the GPU of Exynos chipsets on the Galaxy S4, it blew up into a larger investigation of the practice among many of the mobile manufacturers at the time - with all of them being found guilty. The Samsung case finally resulted in a winning $13.4 million class-action lawsuit verdict against the business — AnandTech, a tech analytics website, was the first to notice the discrepancy and was even included in the court filing.

Over the years, the naming and shaming worked, as merchants rapidly abandoned such tactics for fear of media reaction - the drawbacks significantly surpassed the rewards.

In 2017, Apple was also caught in this manipulative practice, no they did not cheat on benchmarks, Apple has been fined 25 million euros (£21 million, $27 million) for intentionally slowing down older iPhone models without informing customers.

The DGCCRF, France's competition, and fraud watchdog levied the penalties, claiming that customers were not informed. Apple confirmed that it did slow down some iPhones, but said it only did so to "prolong the life" of the devices. the company confirmed it did slow down some models as they age, but not to encourage people to upgrade.

It claimed that as the lithium-ion batteries in the gadgets aged, they became less capable of meeting peak current demands. As a result, an iPhone may shut down abruptly to safeguard its electronic components.

After that iPhone 6, iPhone 6s, and iPhone SE received a software upgrade that "smoothed out" battery performance. A client verified the practice after posting performance tests on Reddit, claiming that their iPhone 6S had slowed significantly as it aged, but had suddenly speeded up again once the battery had been replaced.

But every OEM did not get the same clean chit from users, OEMs of all sizes (including Samsung, HTC, Sony, and LG) took part in this arms race of attempting to fool users without getting caught. 

One investigation on the OnePlus 3T caught that OnePlus was targeting these benchmarks by name, and was entering an alternate CPU scaling mode to pump up their benchmark scores.

Oneplus at that time assured they will not again target any benchmark and will not use this coded manipulation in cheating. But without any such regulations and legal implications. But again in the case of Oneplus 5, they went one step further, the cheating mechanism is blatant and aimed at maximizing performance, unlike last time which did not increase scores by much on average, but did reduce variance and thermal throttling.

. XDA Developers literally put the headlines in their posts, “ [UPDATED STATEMENT] Do NOT Trust OnePlus 5 Benchmarks in Reviews – How our Review Unit is Grossly Cheating at Benchmarks”.

And ..Again… With the Oneplus 9, they cheated in the benchmark again! Anandtech in their test discovered weird behavior on the OnePlus 9 Pro flagship: Popular apps are performance limited, while benchmarks are unaffected. After repeated incidents, year after year Performance benchmarking platform Geekbench, at last, delisted the OnePlus 9 and the OnePlus 9 Pro.  Geekbench officially stated, “ It's disappointing to see OnePlus handsets making performance decisions based on application identifiers rather than application behavior. We view this as a form of benchmark manipulation. We've delisted the OnePlus 9 and the OnePlus 9 Pro from our Android Benchmark chart.”

In 2018 Huawei P20, Huawei Nova 3, and Honor Play were delisted from 3DMark for benchmark cheating.

OPPO Find X and F7 were delisted from 3DMark for benchmark cheating in that year only,

But the big fish caught in 2020, when the SOC maker Mediatek get caught cheating and yes, these chips were in most of all Chinese OEM smartphones, MediaTek was cheating on benchmarks by creating a whitelist of applications that would run the device in a special ‘Sports Mode’ that significantly boosts the performance of a device beyond it’s real-world day to day capabilities and would be unsustainable in day to day usage. Shockingly the cheating mechanism had been hiding in plain sight for years in some innocuous-looking code. Hidden in the firmware there is a .xml file with a list of popular applications with various power management tweaks, including a list of popular benchmarks that would kick off MediaTek's Sports Mode.

 After the article came out about Mediatek malpractice, first of all, they denied the allegation directly. Their official press release says, ‘We believe that showcasing the full capabilities of a chipset in benchmarking tests is in line with the practices of other companies and gives consumers an accurate picture of device performance’’

Other SoC vendors and smartphone vendors that got caught “cheating” in the past have owned up to their behavior and cleaned up their acts.  MediaTek doesn't seem to back down from this at all. In fact, the company published a blog titled ‘Why MediaTek Stands Behind Our Benchmarking Practices’ on their website shortly after the Anandtech article on catching Mediatek malpractice went live.

And all comes down to this 2022, February-March Samsung Mobile.

the company is accused of throttling 10,000 Android apps—but not benchmark apps. It sounds like the scheme OnePlus was caught running last year. Instead of boosting the SoC speeds when a benchmark app is running, Android OEMs are now turning down phone performance any time a benchmark app isn't running. It's like benchmark cheating but in reverse.

Samsung's throttling app is called the "Game Optimising Service." 

Users of the Korean message board Clen.net found wildly different benchmark scores depending on whether benchmark apps had their original names or not.

The list also includes every popular third-party app you can think of—Netflix, Disney+, TikTok, Facebook, Twitter, Amazon, among others. The only apps you reliably won't find on this list are benchmarking apps. Geekbench, 3D Mark, PCMark, GFXBench, Antutu, CPDT, and Androbench are all missing from the list.

And the result was not so much as easy criticism, Geekbench accuses Samsung of benchmark manipulation, delisting the last four years of Galaxy flagships.

Regulatory Demand : 

Unfortunately, these smartphone companies(OEMs) and SOC makers are not bound to not do this malpractice or cheating by any law not in India nor in any specific law. Benchmark cheating, Performance throttling, and performance push in certain actions and apps are not defined by the state or any common stakeholder bodies. We can say these make companies look bad in the eyes of consumers and reviewers, but this reputation loss can be the right redressal method. These malpractices by OEMs and smartphone chip makers are clear manipulation to trick consumers to buy into their products-services which should come under the Consumer Protection Act,2019 in India and also by whitelisting benchmark apps or some specific apps are an abuse of power and position and raises antitrust concern for rest of the apps and even competitive OEMs it should be taken into consideration By Competition Commission of India under Competitions Act. There is a massive regulatory void in regulating and controlling these under the hood CPU-GPU level cheating or antitrust actions by these companies for over 10 Years. It is now time for the stakeholders and government should take note of this situation more seriously and actively, GeekBench or 3D Mark Delist can be the investigation starter but there should be proper procedural safeguards for consumers and competitors and a proper checking process and regulating structure with legal statutory backing. I hope my short article can be the start to the end of mischief by these smartphone companies ( OEMs & SOC makers)

BTS: Thank You AnandTech & XDA Developers for being the Angels in Demon.