Adaptive Training -- How long does a beta usually last?

It seems to be taking a long time to work out the kinks, no?

Feels more like an alpha, as features are changing. A public beta is considered by all my software friends to be a release candidate that would benefit from more widespread testing.

Hoping the final feature list is getting closer, and it leaves beta. But not a good idea to rush it to market.

3 Likes

Based on experience with Machine Learning at work, I would expect AT to remain in beta for significantly longer. TR is trying to solve several problems at one time, and the way theyā€™ve structured the use of AT has - in my opinion only - made their life harder. That is, if TR had constrained the initial use of AT to just people who are only:

  • Using plan builder
  • Doing all rides indoors on the trainer via TR app

Then I think TR would have made faster progress on the base use case - AT being able to adapt plans in a way that improves performance. Now the whole definition of how you measure (or even what to measure) for ā€œimprove performanceā€ is a whole other kettle of fish.

But the way I see AT being used is very unconstrained:

  • Mix of indoor / outdoor TR workouts
  • Mix of structured workouts / unstructured rides (or at least not structured in a way that is easy for AT to identify)
  • Mix of TR created workouts with ā€œapprovedā€ rating & workout creator workouts with dubious / inconsistent rating

Throwing all of these into the mix has, again in my opinion, created a lot of noise that is going to make it much more difficult for the TR AT team to figure out if the way that AT is learning even solves the problem they are trying to solve: use AT as a way to adapt TR Plan Builder plans in a way to make them more effective (again, how is this defined) for the N=1 case

Edited to correct my bad typing

10 Likes

I could be wrong as I am definitely not TR staff, but to me the only thing holding it back in Beta is the inability to process unstructured outside rides (even the structured ones I think rely on your survey input).

Beta definitions shift greatly depending on project and industry. One (relatively) famous example is the Gmail open beta. I think it lasted something like 5 years and had many millions of users before it finally left beta

In other places betas are as described by @bbarrera - a near-final build that is really looking for stress testing and painting the final corners of functionality

All that to say - I wouldnā€™t be surprised if AT is in ā€˜betaā€™ for at least another 3 months, possibly 6. Total guess, could be changed tomorrow

Ultimately, whether or not it is deemed beta is relatively meaningless from a user perspective at this point. That may change if they decide to roll back functionality, but for most users, and the most likely path forward for the application - beta or officially released is irrelevant

4 Likes

I would also add being able to adapt it to the triathlon plans, which Iā€™m sure itā€™s a relevant part of their userbase. Right now, it would swap my brick runs (designed to be 15-20 min long) for much longer runs, as it doesnā€™t understand what a brick session is. Similar for the rides also, when Iā€™m supposed to ride continuously at tempo, it would swap it for workouts with rest periods inbetween.

3 Likes

Thereā€™s still a long way to go. I think for many (most?) people itā€™s not a fully workable product even as a beta until it can handle unstructured rides.

3 Likes

Iā€™m not following a triathlon plan so I missed that, sorry.

Provided youā€™re doing 2-3 indoor TR rides a week (or functioning TR outdoor workouts, wherever that is these days), then I think the ability to consider unstructured rides is a bit of a red herring for a large % of users.

Thatā€™s basically how I have been using AT and it worked out great for me. E.g., one VO2 and one Threshold workout a week is relatively normal on TR plans - if they happen to get easier cos I have done loads of VO2 on club rides etc, then my survey responses change and AT bumps up the next workouts.

Thereā€™s a couple of times where TR suggests workouts that are way easy just because I hadnā€™t done anything in TR in those zones recently. Especially Endurance. Thereā€™s 2 ways around this - manually associate TR endurance rides with your unstructured outside rides in order to keep the PL up, or just pick a harder endurance ride from the Alternatesā€¦

I personally donā€™t have a big issue with picking alternates from time to time but I gather some people do. Obviously, the people on here are likely to be in the vocal minority so I am not sure if forum feedback on AT is really representative of the user base, A lot of users will just pick it up and use it.

7 Likes

realizing Iā€™m not an edge case, I did an entire base-build-specialty cycle with AT and it worked flawlessly. I donā€™t really do unstructured stuff, so on a couple of occasions I did (a fast group ride and a century) I didnā€™t get progression level changes, but in my opinion itā€™s inconsequential unless youā€™re actually doing a lot of work on particular energy systems and then you get served really easy workouts relative to your abilities. The easy step is to try a level 4 workout and help AT with setting a better baseline level. With the current fix to outdoor workouts to auto-pass, I canā€™t imagine too many reasons not to keep up training. And for those who just ride for fun and donā€™t want to do structured training, well thatā€™s the tradeoff, either go out and do whatever or stick with a plan.

5 Likes

except for the recent regression to not update PLs if its a custom workouts.

How many TR users do high-volume full based/build/specialty cycles, and almost all on the trainer? Maybe you are an edge case?

1 Like

I actually meant that I am an edge case, not not lol

But Iā€™ve done plenty of my workouts outside this year too, even prior to implementing the current fix, and I had no issues running workouts and getting credit.

1 Like

Iā€™m with @trpnhntr on this, I assume most TR users are probably on AT now so changing it to from beta to ā€œfull releaseā€ doesnā€™t really change anything because even if itā€™s out of beta you can be sure itā€™ll be updated/changed/improved for a long time to come. The AT process is simply too new for the coaching/training industry, so the more AT data they have to analyze the better they can refine the AT progression.

However I canā€™t wait for progression levels to figure out unstructured group rides/races (obviously a very tough challenge), and for the levels not to be so sensitive to small-ish FTP changes.

Having done some alpha and beta testing for a very, very big software company, a public beta need not be close to a release candidate. Apple and Microsoft release various types of betas of their software. I would liken TRā€™s closed beta to the developer betas Apple distributes of its OSes. Of those, some get designated Public Betas. But early public betas need not be close to the release candidate. When I was testing for Apple, a release candidate was almost always almost identical to the golden master (i. e. the final initial release) unless there were showstopper bugs, i. e. the finished product released to all customers.

Google famously released quite a few features and products as beta even though other companies would just call them initial releases. Gmail started as a public beta in 2004. If memory serves, it stayed a ā€œbetaā€ for many years. Then there is software that is not considered beta, but extremely broken (btrfs comes to mind).

Drawing from my experience, the closed beta felt like developer betas: broadly speaking, the features were there, but you ran into lots of edge cases. The open beta right now feels like a public beta with some important features being delayed.

Most notably is the algorithm that scores outdoor rides and outdoor workouts (at least those done with Wahoo head units). They can screw up PLs of people who ride a lot, and there is no good solution right now, but to override AT when necessary. I reckon that this algorithm might also be used to score custom indoor workouts. (Last time I checked, the workout editor is still based on Adobe Flash, which has been deprecated years ago and looks like it has been designed for Windows XP in 2007.) However, you can mitigate this by getting to know AT and overriding it when it makes sense. I donā€™t find that super hard, to be honest, and the progression levels are on average a very good indicator of how hard a workout taxes you in a particular power range.

I wouldnā€™t mind if AT stayed in public beta for quite a while longer: it is a feature that can only get better if it is used and tested by a wide array of athletes ā€” including people like you who like to rock their custom workouts. :slight_smile: (I reckon that alone makes AT of much less use to you than to me.) And your criticism that you donā€™t find AT useful for you is totally fair. Fortunately, the solution to your issues is also clear ā€¦ but unfortunately, isnā€™t on the horizon yet.

IMHO my main beef with TRā€™s software is elsewhere: beyond the workout screen (which does its job very well), I just donā€™t find their apps very good. One big miss is the lack of analysis tools: how did my performance change over the course of one or several seasons? How does my performance compare the same time last year? The closest is the list of dates and FTP numbers, but I donā€™t think anyone can parse this without creating a graph. They should also integrate some basic nutrition info (optionally, of course) so that you can track how much you drank or ate during workouts and compute liters per hour or grams of carbs per hour automatically. Things like sleep tracking and the workout editor should be built in. Integration with iOSā€™s Health app. Similarly, I think you should have some visual marker if you took a break during a workout. Oh, and proper calendar management inside the mobile apps.

4 Likes

Yes. And Iā€™ve got a long wish list too. My fundamental issue is that I found a better macro approach, or base/build/specialty if you will, and no amount of AT is going to fix that. Iā€™m absolutely certain TRā€™s approach works for others, but I only care about my performance. The season planning discussion in HighNorth guide (that I bought today) had an interesting discussion on season planning.

4 Likes

IMHO TR puts some effort into displaying how your fitness develops over time, AT could be very useful at one point. Progression levels could track your progress. And you can enter the FTP by hand, so you can use any testing protocol youā€™d like.

And you might still benefit from AT if you pick your workouts for your custom training plan from TRā€™s catalog. Progression levels should still give you a good indication of your progress, and be a good indicator of what a good alternate workout is.

Yes, TR has had the ability to show how fitness increases or decreases over time. Its called a power curve and was rolled out when performance analytics was launched over 29 months ago.

Lets look at how my fitness dropped after switching from self coaching to using TR training plans:

and then increased by switching from TR to another approach using an off-the-shelf plan (purchase once, use over and over):

That is 7 months of FasCat off-the-shelf plans with some mods based on feeling. Really I stopped training the second week of June, most of the gains on that chart are from 5 months of switching to FasCat approach.

And some improvements on short power too, after switching from TR to FasCat approach:

Thanks for the pep talk, but the data and feelings are crystal clear. Iā€™m sure adaptive training would help, thats not the point. Using other analytics I can post my heart rate data, and this new approach has put less strain on my heart.

Thanks but Iā€™ll stick with what is working, as Iā€™m stronger and healthier by following a different road.

I know about the power curve, but it provides only very coarse grained information (not least because it is based on seasons). Two big problems that come to mind: (1) you will have to wait for a while for it to be populated with meaningful data and you have close to zero granularity in the time dimension. And (2) what if your goals change from one season to the next? To give you an idea what I mean here, last season I opted for the Crit Plan to change things up and I was hoping to compete in a few crit races (I did one so far). As a result, I got a massive boost in FTP and could hold 117 % FTP for almost 7 minutes without going all-out. But I predictably I wasnā€™t able to hold my (relative!) power level for as long as I did before. That wasnā€™t because I was failing, that was by design. In absolute terms, I had the best season ever. How do I extract that kind of nuance out of my power curves? What if I want to check progress mid-season? Furthermore, it is currently not available on the iOS app.

All I am saying is that what TR has now is not, hmmm, good. A chart with progression of PLs on top of FTP changes could add very useful information here. But so far they havenā€™t attempted that. Heart rate data could be very useful to show how fit you are. (I can tell how fit I am by seeing how quickly my heart rate recovers after a hard interval to my VT1ish heart rate. If it is less than 1:30 minutes after a few intervals, Iā€™m in good shape.)

I think you misunderstand my arguments: I think the tools TR makes should also serve people like you who are probably using Training Peaks in addition to TR (Iā€™m guessing here, so feel free to tell me that I am wrong).

I used to use TP, but found their system of their trifecta of metrics antiquated. It was less a measure of my performance and more a measure of how hard/easy a training week would probably feel like. Their race and ride analysis tools were nice and you could track metrics like mood and sleep, but in the end I wasnā€™t using it enough to really justify the cost. Plus, I felt like they werenā€™t doing anything with the data, Iā€™d have to parse them manually.

1 Like

Iā€™m not buying the ā€œpower curve, but ā€¦ā€ argument. My entire power curve dropped after switching from self-coaching with Carmichael Training Systems approach over to TR plans. Then my entire power curve increased after switching from TR plans to FasCat approach. Iā€™m faster, stronger, and healthier. Without question, except perhaps in forum-land :man_shrugging: In the second chart above I didnā€™t even train 2-5 minute power and saw a huge increase over TR.

Iā€™ve got a WKO license (paid $170 Sept 2019) and can extract all kinds of interesting and nuanced stuff. Except for things like yesterday, wherein at the start of my 4th week back after a 5-week off-season (during which I was sick twice), my 2 hour benchmark zone2 workout is back to the peak of my fitness in May. But the same happened when I was self-coaching in 2017, and I have a hunch why.

???
I am confused why you interpreted my post as criticism of your training approach. I did not. You should do what works for you, and it seems you have found your jam. I just think TR should aim to cater people like you who use their own or external training plans better than they currently do, not least because many other users could benefit from the additional functionality. Specifically, I was criticizing that TR did not put in much effort into showing you how your capabilities evolve over time. (And at least when I used it neither did TP.)

Yes, it is easy to see improvements if all of your numbers increase, so you could easily see your progress in the power curve. But I think my point still stands: the power curve only gives you one particular way to visualize your performance data, and it does not have much resolution in the time direction. Nor are you able to answer more refined questions.

Iā€™d like to tease out more nuanced information like ā€œHow many watts/%age of FTP did I lose over my rest period?ā€ ā€œHow did my fitness develop after, say, the first block of Base training? What workouts did I have a hard time with last season? Did I achieve my last seasonā€™s goals (e. g. improving my short power), i. e. did e. g. the short power build block affect my capabilities differently than the general build block I did the year before?

2 Likes