Add TW3Cast results - #209
nattanioCalor wants to merge 1 commit into
Conversation
|
Thanks for the contribution! Unfortunately we can't verify the commit author(s): Nathan <n***@t***.com>. One possible solution is to add that email to your GitHub account. Alternatively you can change your commits to another email and force push the change. After getting your commits associated with your GitHub account, sign the Salesforce Inc. Contributor License Agreement and this Pull Request will be revalidated. |
e3b9551 to
b9ba073
Compare
|
Hi @nattanioCalor, Thanks for the submission! Frankly this is the first time we get a mixture of routing + finetuning submission so I am not sure about what to label it, but FYI: I started working on replicating the results. |
|
Hi @nattanioCalor, Separately on replication I tried to give it a go on a smaller subset and realised that certain parts of the code is not released? For bizitobs_application/10S/{medium,long}, the public router specifies tournament, but the repo does not provide the tournament code and winners are not published (please correct me if I am wrong). Any suggestions how to proceed? Moreover I tried m4 weekly which does not require a tournament:
I followed the public TW3-Cast README and used the published expert checkpoints and router configuration, I used the published frozen Toto/TimesFM route and got ~0.03 diff. Would appreciate your help to address these problems. It would make life much easier for me if the repo had all code available and a single entrypoint that lets me replicate any dataset. Thanks again! |
Adds results for TW3Cast (TW3 Partners): a frozen per-configuration router over lightly fine-tuned experts (Chronos-2, TiRex, Toto 2.5B LoRA), no agent and no LLM at inference, no test-data leakage in any component. Scores computed with the official harness (gluonts evaluate_forecasts, leaderboard parameters). Code and router table: https://github.com/TW3-Partners-OS/TW3-Cast