This report presents clear evidence, based on robust data, that power readings derived from dual sided cycling power meters using Shimano cranks, does not provide consistent levels of accuracy as specified by power meter manufacturers.

The Shimano alloy crankset is a popular base unit for a number of companies to use for their power meter products. Not originally designed as a power meter, electronics engineers successfully mounted strain gauges on the left side crank only (where power was doubled) to give cyclists a more cost effective way of measuring their power output.

Stages Cycling was the first company to develop left-crank based power, applying strain gauges to the crank arm. Stages Cycling – Wikipedia 

Following the success of Shimano based left-only power meter products almost every company started offering a dual sided power meter configuration by placing strain gauges on the corresponding right side crank arm.

From a consumer perspective this was welcomed. A somewhat cost effective (“somewhat” as it’s not as cheap as a left-only offering) dual sided power meter, independently measuring left and right power, and reporting metrics such as pedal balance and true total power output. 

From an engineering perspective the right side Shimano crank power meter has been a challenge. The design and shape of the right crank is very different from the left, and with Shimano changing the shape from the FC6800/FC7900 with the R8000 and R9100 right side cranks for a wider four arm arm/spider interface, things became even more challenging for them to obtain accurate power (torque) measurements from. There has been mention of a strain gauge redesign required specifically for the R8000 & R9100 right side cranks.

Over the past year I’ve ridden and compared power readings from ELEVEN (11) Shimano R8000/R9100 cranksets from seven companies. I’ve had other riders perform the same testing protocols on a number of these units. Based on the data from these tests, and confirmation of these results from independent electronic engineers who are familiar with cycling power meters – I do not believe the power measurement from any Shimano R8000 or R9100 RIGHT side cranks is at an acceptable level for consumers to confidently rely on at this point in time.

I have been in direct communication with a number of the companies mentioned in this report over the last 12 months.

None of these companies have questioned my testing protocol, or results for what would be considered successful tests of left-only power meters and their non Shimano dual sided power meters.

Each company has been provided this report and all data collected on thier power meters in advance for response.   

My goal here is to identify issues consumers may encounter themselves with these products. I hope these findings will lead to more accurate power meters on the market.

Background

Strain gauges are a solved problem. The gauges used today are reliable and dependable. Other power meter features are of a high standard: Battery life. ANT+ & Bluetooth Smart data transmission. Active Temperature Compensation (to an extent). If this wasn’t the case the power meter industry would be in trouble.
 
Digital cadence sensors are also typically accurate enough to obtain quality power output from. (Accurate cadence is required for accurate power from these power meters). Some power meters still rely on magnets for better cadence readings (which are more responsive in my experience). Accurate cadence readings are a part of this bigger picture discussion, but they’re not the main show in this report.

Why are power meters so hard to get right? 

Bicycle power meters are effectively a moving weight scale. Standing on a bathroom weight scale requires you to be stationary for accurate results. Power meters require the same accuracy moving at high speed (rotation, vibrations, etc), while measuring multiple force vectors, in almost all environmental conditions imaginable.

Shimano Cranksets as Power Meters

Shimano produce cranksets, not power meters. Converting these cranksets into power meters is a post-production process where strain gauges are typically placed on the external surface of the crank arms.

Shimano use the same post-production strain gauge placement design as third parties for their DuraAce R9100P Power Meter. However when Shimano themselves can’t produce a quality industry leading power meter from their own cranks from day one – there’s a problem. A big problem. 

References:
– DCRainmaker DA9100P Review: Issues with left/right accuracy.
– Bike Radar DA9100P Review (Ben Delaney): Issues with left/right accuracy.
– GPLama DA9100P Review : Issues with left/right accuracy.

Discussions with a power meter designer/engineer Keith Wakeham on the topic of using Shimano cranksets as power meters has confirmed there are difficulties with design asymmetry, chainring positional affinity, and the coupling effect when using strain gauges on the RIGHT side crank arm of a Shimano crank. This is a known problem that isn’t limited to just the drive side, however it’s where the issues arise when using a DUAL sided Shimano based power meter.

Inside Shimano HOLLOWTECH Cranks

Onto the testing…

The Lama Lab Test Protocol: 

My “Lama Lab” test protocol has shown to provide quality results when testing both power meter and smart trainer power measurement. It’s by all means not perfect, however it has successfully identified issues that have been acknowledged (and in some cases resolved) by power meter and smart trainer manufacturers.

This protocol uses a combination of SIM (Simulation) and ERG mode. ERG mode forces you to pedal at the set target wattage. For more details on ERG mode I have a 12 minute video explaining ERG over YouTube: ERG Mode Explained.

– Power meters stationary in the room for at least 10 minutes (temperature adjustment)
– Data recording on Garmin Edge or Wahoo Elemnt units (using the same vendor for all meters if possible). 
– 1 second record interval. GPS disabled. Units placed <1m from power source.
– Clean chain (<0.75% wear). Clean Cassette.
– Straight Chainline selected (36/39 17-18) for ERG.
– Zero-offset performed on all units (as applicable) at 0 minutes.
– 10 minute ride (SIM mode / Human warm-up / a few short hard efforts).
– Dismount bike. Zero-offset performed on all units / Spindown on trainer (if applicable).
– 10 minute steady-state ERG @200W ~88-95rpm. Both hands on bars. Natural/smooth pedalling. 
– 10 minute steady-state ERG @250W ~88-95rpm. As above. Balanced, smooth, natural pedalling.
– Maximal sprint of 5-8sec. Out of saddle.  (SIM or Level mode).
– 4 minutes easy spin (SIM or Level mode). 
– Over/Under Intervals in ERG each of 20 seconds: 150W/350W/150W/350W/150W/450W/150W/450W/150W. Balanced, smooth, natural pedalling through these intervals. 
– Cool-down. Sometimes involving a harder 350W/400W effort for ~60 seconds in SIM to collect more data. 
– All data (FIT format) labelled according to equipment used and uploaded to DCR Analyser.

Shimano Based Power Meter Cranks: LEFT ONLY

Data shows left-only Shimano based power meter cranks typically report power within their specified accuracy specification (+/-%).  I have encountered some left side cranks reporting lower than expected power measurements. In some instances this has been resolved with re-installation and correct torque on the pinch bolds (14Nm – max specified by Shimano). I have had left-only cranks from manufacturers not listed in this report that have zero-offset stabilisation issues and power fluctuations. So they’re not immune to issues either.

The caveat with LEFT only is with the doubling of power from one side. It assumes 50/50 balance for the unit to show “correct” output. Small lag in power reporting due to no measurement on right side. 

In short – Data collected on LEFT only Shimano crank power meters appears mostly accurate. This report focuses on the RIGHT side crank arm.

References:
Stages Power Meter (Single/Gen III) – Install, Ride, Data Review
Magene RIDGE Shimano Ultegra R8000 Power Meter: Details // Install // Data Review
XCadey X-Power Budget Power Meter: Unboxing, Install, Review

Shimano Based Power Meter Cranks: RIGHT / DUAL SIDED

Here’s where things get tricky.

Shimano Cranks Tested (11):  

– Stages LR (x2) 
DuraAce R9100 (Gen 3) 1st Unit
DuraAce R9100 (Gen 3) 2nd Unit

– Pioneer (x3)
DuraAce R9100 (SGY-PM910H ANT+ only model) 
DuraAce R9100 (SGY-PM930H ANT+/BLE model) 1st Unit
DuraAce R9100 (SGY-PM930H ANT+/BLE model) 2nd Unit

– 4iiii (x2)
DuraAce R9100 (PRECISION PRO) 
Ultegra R8000 (PRECISION PRO)

– Giant 
Ultegra R8000 (Power Pro MY19) 

– Shimano 
DuraAce R9100-P 

– WatTeam PowerBeat
Ultegra R8000 G3 (*Gauge placement different to the others) 

– XCadey 2XPower
Ultegra R8000

Method: 
Lama Lab Test as documented above and here: GPLama How To: Comparing Power Meters // Power Accuracy Testing // DCR Analyzer Tool: https://www.youtube.com/watch?v=JhheDLXRM7I

The Results: 

Steady-State 250W ERG  

Power Meter: Crankset used
ERG: 50% of 250W ERG Mode (125W)
Other: Right side power channel of non Shimano Crank (Assioma/P1/Vector3)
Right: Shimano RIGHT side reading

– Variance higher at 350W & 450W ERG.

– Differences not as pronounced in random terrain SIM mode or riding outside. 

– Harder steady-state efforts indoors (SIM) and outdoors does reproduce this issue. 

Isn’t this just drivetrain loss?

No. The discrepancy in power readings is between the pedal power meter and crank power meter which are directly connected. The power differences reported are more apparent on the RIGHT side. The controlling smart trainer in ERG also provides a tertiary power comparison source.

What did I compare against? 

Multiple sources:
– Favero Assioma DUO (Dual sided pedal power meter. ±1% Accuracy) 
– PowerTap P1 & P2  (Dual sided pedal power meter ± 1.5%) 
– Garmin Vector 3  (Dual sided pedal power meter ±1% Accuracy) 
– Tacx Neo / Neo 2 (Direct Drive Smart Trainers ±1% Accuracy) 
– Wahoo Kickr / Kickr CORE (Direct Drive Smart Trainers ±2% Accuracy) 

Basic Formula:
Dual Sided Power Meter L + R = Smart Trainer Power ≠  Shimano Crank Power Meter Total Power.

eg. AssiomaDUO Power = Tacx Neo Power ≠ Shimano Crank Power. 

More specifically indoors and out: Power Meter R ≠ Shimano Right Crank Power.

Note: This doesn’t factor in any specific mechanical losses incurred between the pedal and crank, and drivetrain losses at the smart trainer power measurement point (effectively the rear hub). However, measured losses are never greater than a few watts (<2%) of TOTAL power using a clean drivetrain when cross-chaining using 52/28. Reported power from the RIGHT cranks is up to 10% different, on one side alone.

I can not attribute the differences recorded to mechanical or drivetrain losses.

Why hasn’t this been detected / reported / resolved? 

Comparative testing of consumer power meters isn’t commonly performed. Not typically to the level of detail (and cost) that it has taken me to reach my current conclusions. 

The independent testing of Shimano R9100 LR cranks at the University of Colorado Boulder commissioned by 4iiii on three of their cranks was based on a steady-state riding protocol on a treadmill (not ERG). With their comparison being calculated wattage, not other power meters, and using two minute intervals at lower cadence ranges than I would expect a trained cyclist to ride (only up to 85rpm). Their summary result of 1.58% accuracy was an average across all data sets with the three Shimano R9100 cranks tested.

There was datapoint at ~250W (calculated) reporting as ~240W. Indicating there is outlying data points. 240W @250W calculated is -3.70% equating to -9.7W. What wasn’t reported was the Left/Right power figures on any of these data sets. Yes, I’m being selective here – my point being we need to know more detail as to why that data was out by that much.

On the upside to the testing in Boulder, their results with non Shimano cranks using their strain gauge technology were a LOT better. With the carbon SRAM XX1 crank achieving 0.84% accuracy on average across four cranks tested. Indicating the gauges and technology used by 4iiii is of a high standard that will produce accurate measurements assuming the base crank it is installed on doesn’t suffer from the same problems as the Shimano RIGHT crank design.

Performing the Lama Lab Test protocol on the dual sided FSA SL-K crankset fitted with 4iiii power meter gauges reported no issue with left or right side power readings.

Why haven’t other media and reviewers reported this?

I’ve not seen any reports showing the same testing as what I use in the Lama Lab (as documented above). Specifically longer duration smooth pedalling steady-state ERG efforts in addition to over/under intervals – Tested against a long-term known-good power sources – Which I consider the AssiomaDUO / Tacx NEO / PowerTap P1 to be.

I have discussed this issue in a number of reviews already. In particular the Shimano R9100P, my WatTeam G3 review (results confirmed by WatTeam engineers), and in my Stages LR video. This isn’t ‘new’ news.  

This is also difficult to diagnose. It requires multiple power measurements on the same bike at the same time. Not many cyclists will install two independent L/R power meters on their bike. Even less will perform any in-depth analysis of the data. A growing number of cyclists are comparing their bike power meters to their smart trainers and are reporting power differences. This issue could very well provide answers to some of those queries.

More often than not when users ask questions regarding power meter discrepancies in a public forum they are dismissed as ‘drivetrain loss’ or L/R imbalances. Unfortunately these reasons are echoed by some manufacturers without performing a full analysis of each issue. This is not a drivetrain or a rider L/R imbalance issue.

What exactly is the issue and why hasn’t this been resolved in firmware, or with better strain gauges?

The engineering complexities of this problem is best explained by someone who can cover this in the technical detail it deserves. And that person is Keith Wakeham. Keith is one of the original designers of the 4iiii power meters. Keith is listed on the patent under ‘Inventor’. He really knows his stuff. Reference: https://patents.google.com/patent/WO2016030768A2

Keith has a blog over on https://titanlab.co and has put together a post on this very topic: https://titanlab.co/shimano-crank-design-blunder/

If you have a few minutes to keep diving into this issue to the depths you never thought possible – Grab yourself a coffee and watch his video here:

This isn’t an incorrect slope or miscalibration issue. It’s much deeper and harder to solve given the design of the Shimano RIGHT crank arm/spider

For now this issue exists even with the latest meters with the latest firmware.

Team Sky / Pro Team X / Mine Reads Fine

Professional teams are sometimes provided non commercially available product. Teams are paid to use equipment (read between the lines here of what they’ll say about said product). Professional teams are also unlikely to perform any power comparison tests to the level required.

“Mine is accurate”. Compared to….? What was your testing protocol? How was the data recorded and analysed? The answer here is typically “we’ll need more data”. Beware. Getting to the answer is a rabbit hole of indeterminate depth.

My Take Outs….

Shimano based dual sided power meters report lower total power in my tests due to inaccurate measurement of power on the right side in a number of pedalling scenarios. This is particularly evident in steady-state smooth pedalling efforts across a range of wattages.

At this point I am not confident to use ANY Shimano based dual sided power meter crankset as a baseline to compare power readings from other power meters or smart trainers.

I own a Shimano based dual sided power meter as referenced above. Does this matter? 

Yes:
– If you’re using the reported L/R balance values as part of your training.
– If you’re comparing multiple power sources.
– If you have more than one type of power meter (One that’s Shimano based, and one that is not).

No:
– You’re not bothered with L/R balance. 
– The power meter is your ONE source of power for all training.
– You have more than one of these Shimano crankset power meters.
– You’re not worried about a few % margin of error with your real power output (and metrics that are slightly skewed because of this) 

Company Responses… and where to from here.

All in all I’ve welcomed the discussion this report has lead to with most of the companies named. There is obvious (and admitted) complexities with the Shimano RIGHT crank design when used as a power meter. 

I understand it’s not in the interests for any of these companies to admit any major shortcomings of their products. What I’m looking for is a sound explanation as to why these discrepancies have been narrowed down specifically to Shimano RIGHT cranks – why I’m not able to use any of them as a reliable baseline of power comparisons – and most importantly – is there a possible fix for this?

Shimano Australia:
– Stand behind their ±2% accuracy claims of LEFT and RIGHT Shimano Power Cranks.
– Discrepancies in my provided RIGHT power data due to bugs in the firmware I used.
– I tested the R9100-P with 4.0.7 (March 2018) which is the firmware that their changelog indicates resolves accuracy issues.
– It was recommended I update to 4.1.7 (Feb 2019) and retest.
– I raised concerns that 4.1.7 change log lists no updates to power accuracy. Also infers they had an inaccurate power meter on the market up until February 2019…!?

My Action Points: Source another R9100-P and retest with 4.1.7.

4iiii Innovations: 
– Acknowledged issues with the Shimano crank asymmetry and non-uniform coupling effects.
– They believe their firmware and calibration protocols are robust enough to stand by their accuracy claims. 
– My riding style given as a possible reason for the discrepancies seen in ERG. (This doesn’t explain why my riding style wasn’t a problem with their non Shimino L/R crank in the Lama Lab tests, or Dr. S.Lane also recording the same lower power measurements with the Shimano based RIGHT crank). 

My Action Points: Continue working with 4iiii to provide more data from the Lama Lab and in person across more of their power cranksets (Both Shimano and non Shimano). Ongoing.

Stages Cycling:
– Stages stand behind the accuracy and repeatability of the Stages Power R and Power L/R meters in the vast majority of use.
– They mention the loading of the spider and crank is different during the pedal stroke in ERG as opposed to SIM or riding outside – which could be contributing to the discrepancies. 

My Action Points: Continue working with Stages to provide more data from the Lama Lab and in person across more of their power cranksets (Both Shimano and non Shimano). Ongoing.

Pioneer:
– Data analysis still in progress. TBA. 

Giant: 
– No response. TBA.

WattTeam: 
– No longer in business. RIP.

XCadey:
– Data analysis still in progress. TBA. 

Notes/References:

– In comparative power meter testing it is difficult to use the total ride/activity power as a measurement of accuracy due to factors such as start/stop delays across different power meters, possible data interruption (drop outs), etc, all impacting the overall total power recorded. This is why the focus of this report is specifically on 250W ERG steady-state efforts.

– The data from the Giant PowerPro has changed with recent firmware updates. From reading high on the right, to low, to a random point in between, sometimes. I suspect they are fudging or scaling the left/right to account for these issues. The dataset used here indicating LOW readings on the right was from fw237 (latest release as of testing).

DCR Analyser: Online tool for reviewing and comparing FIT file data. 

Shimano R9100-R Power Meter tested June 23rd 2018 with Firmware 4.0.7 released 27th March 2018 – “Update to make output value more accurate”

Technical Report:Accuracy Testing of 4iiii Innovations PRECISIONPowermeter Technology. Scott Cooper, Ph.D, 4iiii Innovations Inc. Wouter Hoogkramer, Ph.D and Rodger Kram, Ph.D, University of Colorado Boulder University of Colorado Boulder Locomotion Lab September 14, 2018

Patent US10060738B2 (United States) Adhesively coupled power-meter for measurement of force, torque, and power and associated methods Inventor: Kipling Fyfe, Keith Wakeham

Validity and Reliability of the PowerTap P1 Pedals Power MeterJ Sports Sci Med. 2018 Jun; 17(2): 305–311.