Skip to content

Comments

[TimesFMv1] fix variance calculation#341

Merged
rajatsen91 merged 2 commits intogoogle-research:masterfrom
kashif:timesfmv1-var
Feb 19, 2026
Merged

[TimesFMv1] fix variance calculation#341
rajatsen91 merged 2 commits intogoogle-research:masterfrom
kashif:timesfmv1-var

Conversation

@kashif
Copy link
Contributor

@kashif kashif commented Nov 25, 2025

  1. Masked variance calculation (lines 95-107): Changed from the numerically unstable E[X²] - E[X]² formula to the stable centered formula E[(X-μ)²]
  2. Sigma clamping (line 609): Changed from torch.where(sigma < tolerance, 1.0, sigma) to torch.clamp(sigma, min=tolerance)

by: huggingface/transformers#42099 @AnMakc

  1. Masked variance calculation (lines 95-107): Changed from the numerically unstable E[X²] - E[X]² formula  to the stable centered formula E[(X-μ)²]
  2. Sigma clamping (line 609): Changed from torch.where(sigma < tolerance, 1.0, sigma) to torch.clamp(sigma,   min=tolerance)
@AnMakc
Copy link

AnMakc commented Nov 25, 2025

@kashif
Thanks for bringing it here.
Could you also fix the jax version, it has the same issue:

masked_var = masked_squared_sum / num_valid_elements - masked_mean**2




Btw, this potentially affected model pre-training as well.

@kashif
Copy link
Contributor Author

kashif commented Feb 18, 2026

cc @siriuz42 for your consideration

Copy link
Collaborator

@rajatsen91 rajatsen91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL

Copy link
Collaborator

@rajatsen91 rajatsen91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok lets go ahead, thanks.

@rajatsen91 rajatsen91 merged commit 8a755c9 into google-research:master Feb 19, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants