js/ml/metrics

js/ml/metrics/index.ts

fino:ml/metrics — shared scoring for classification, ranking, regression, calibration, and vector similarity.

One implementation of every standard metric, so that evaluation harnesses, classical estimators, and inference pipelines all report the same number for the same data instead of each growing a private scorer. Everything here is pure TypeScript over ArrayLike<number> and primitive labels — no tensors, no native dependencies, and safe to call from DataLoader workers.

Choosing a metric

accuracy answers "how often is it right", but on imbalanced data it flatters a model that only predicts the majority class — reach for balancedAccuracy, f1Score, or matthewsCorrCoef instead. precision and recall split the two ways a classifier fails, and fBetaScore weights one against the other when a miss and a false alarm cost different amounts.

Metrics over scores rather than hard predictions need no decision threshold: rocAuc summarizes ranking quality across all of them, and averagePrecision does the same while ignoring true negatives, which is the honest choice when positives are rare. logLoss and the calibration family ask a different question again — not whether the ranking is right, but whether a predicted 0.9 actually happens 90% of the time.

ConfusionMatrix underpins the classification metrics and is worth using directly whenever more than one of them is needed, since it derives them all from a single pass.

import { ConfusionMatrix, f1Score, rocAuc } from 'fino:ml/metrics';

const yTrue = [1, 0, 1, 1, 0, 1];
const yPred = [1, 0, 0, 1, 0, 1];
console.log(f1Score(yTrue, yPred).toFixed(4)); // 0.8571
console.log(ConfusionMatrix.from(yTrue, yPred).accuracy().toFixed(4)); // 0.8333
console.log(rocAuc(yTrue, [0.9, 0.1, 0.4, 0.8, 0.2, 0.7])); // 1

Streaming and sharding

Batch functions need the whole dataset in memory. The Streaming* accumulators do not: they hold constant memory, take one batch at a time, and merge across shards, so a metric computed by parallel realm workers equals the one computed serially.

import { StreamingConfusionMatrix } from 'fino:ml/metrics';

const shardA = new StreamingConfusionMatrix();
shardA.updateAll([1, 0], [1, 0]);
const shardB = new StreamingConfusionMatrix();
shardB.updateAll([1, 1], [0, 1]);
console.log(shardA.merge(shardB).value().accuracy()); // 0.75

Malformed input — mismatched lengths, empty arrays, probabilities outside [0, 1], an unresolvable positive label — throws MetricError. Genuinely undefined values, such as the precision of a class that was never predicted, are reported as 0 rather than NaN.

Functions

function accuracy(yTrue: ArrayLike<Label>, yPred: ArrayLike<Label>): number

Re-exported from classification.accuracy.

function averagePrecision( yTrue: ArrayLike<Label>, scores: ArrayLike<number>, options: ProbabilityOptions = {}, ): number

Re-exported from classification.averagePrecision.

function balancedAccuracy(yTrue: ArrayLike<Label>, yPred: ArrayLike<Label>): number

Re-exported from classification.balancedAccuracy.

function cohenKappa(yTrue: ArrayLike<Label>, yPred: ArrayLike<Label>): number

Re-exported from classification.cohenKappa.

function f1Score( yTrue: ArrayLike<Label>, yPred: ArrayLike<Label>, options?: AverageOptions & { average?: Exclude<Average, 'none'> }, ): number

function f1Score(
  yTrue: ArrayLike<Label>,
  yPred: ArrayLike<Label>,
  options?: AverageOptions & { average?: Exclude<Average, 'none'> },
): number
function f1Score(
  yTrue: ArrayLike<Label>,
  yPred: ArrayLike<Label>,
  options: AverageOptions & { average: 'none' },
): number[]

Re-exported from classification.f1Score.

function fBetaScore( yTrue: ArrayLike<Label>, yPred: ArrayLike<Label>, beta: number, options?: AverageOptions & { average?: Exclude<Average, 'none'> }, ): number

function fBetaScore(
  yTrue: ArrayLike<Label>,
  yPred: ArrayLike<Label>,
  beta: number,
  options?: AverageOptions & { average?: Exclude<Average, 'none'> },
): number
function fBetaScore(
  yTrue: ArrayLike<Label>,
  yPred: ArrayLike<Label>,
  beta: number,
  options: AverageOptions & { average: 'none' },
): number[]

Re-exported from classification.fBetaScore.

function logLoss( yTrue: ArrayLike<Label>, probabilities: ArrayLike<number>, options: ProbabilityOptions & { eps?: number } = {}, ): number

Re-exported from classification.logLoss.

function matthewsCorrCoef(yTrue: ArrayLike<Label>, yPred: ArrayLike<Label>): number

Re-exported from classification.matthewsCorrCoef.

function precision( yTrue: ArrayLike<Label>, yPred: ArrayLike<Label>, options?: AverageOptions & { average?: Exclude<Average, 'none'> }, ): number

function precision(
  yTrue: ArrayLike<Label>,
  yPred: ArrayLike<Label>,
  options?: AverageOptions & { average?: Exclude<Average, 'none'> },
): number
function precision(
  yTrue: ArrayLike<Label>,
  yPred: ArrayLike<Label>,
  options: AverageOptions & { average: 'none' },
): number[]

Re-exported from classification.precision.

function precisionRecallCurve( yTrue: ArrayLike<Label>, scores: ArrayLike<number>, options: ProbabilityOptions = {}, ): PrecisionRecallCurve

Re-exported from classification.precisionRecallCurve.

function recall( yTrue: ArrayLike<Label>, yPred: ArrayLike<Label>, options?: AverageOptions & { average?: Exclude<Average, 'none'> }, ): number

function recall(
  yTrue: ArrayLike<Label>,
  yPred: ArrayLike<Label>,
  options?: AverageOptions & { average?: Exclude<Average, 'none'> },
): number
function recall(
  yTrue: ArrayLike<Label>,
  yPred: ArrayLike<Label>,
  options: AverageOptions & { average: 'none' },
): number[]

Re-exported from classification.recall.

function rocAuc( yTrue: ArrayLike<Label>, scores: ArrayLike<number>, options: ProbabilityOptions = {}, ): number

Re-exported from classification.rocAuc.

function rocCurve( yTrue: ArrayLike<Label>, scores: ArrayLike<number>, options: ProbabilityOptions = {}, ): RocCurve

Re-exported from classification.rocCurve.

function brierScore( yTrue: ArrayLike<Label>, probabilities: ArrayLike<number>, options: { positiveLabel?: Label } = {}, ): number

Re-exported from calibration.brierScore.

function calibrationCurve( yTrue: ArrayLike<Label>, probabilities: ArrayLike<number>, options: CalibrationOptions = {}, ): CalibrationBin[]

Re-exported from calibration.calibrationCurve.

function expectedCalibrationError( yTrue: ArrayLike<Label>, probabilities: ArrayLike<number>, options: CalibrationOptions = {}, ): number

Re-exported from calibration.expectedCalibrationError.

function maximumCalibrationError( yTrue: ArrayLike<Label>, probabilities: ArrayLike<number>, options: CalibrationOptions = {}, ): number

Re-exported from calibration.maximumCalibrationError.

function averagePrecisionAtK( relevance: ArrayLike<number>, k: number, options: { totalRelevant?: number } = {}, ): number

Re-exported from ranking.averagePrecisionAtK.

function dcgAtK(gains: ArrayLike<number>, k: number, options: GainOptions = {}): number

Re-exported from ranking.dcgAtK.

function hitRateAtK(relevance: ArrayLike<number>, k: number): number

Re-exported from ranking.hitRateAtK.

function meanAveragePrecisionAtK( relevances: ReadonlyArray<ArrayLike<number>>, k: number, ): number

Re-exported from ranking.meanAveragePrecisionAtK.

function meanReciprocalRank(relevances: ReadonlyArray<ArrayLike<number>>): number

Re-exported from ranking.meanReciprocalRank.

function ndcgAtK( gains: ArrayLike<number>, k: number, options: GainOptions & { idealGains?: ArrayLike<number> } = {}, ): number

Re-exported from ranking.ndcgAtK.

function precisionAtK(relevance: ArrayLike<number>, k: number): number

Re-exported from ranking.precisionAtK.

function rankedRelevance<T>(ranked: ArrayLike<T>, relevant: Iterable<T>): number[]

Re-exported from ranking.rankedRelevance.

function recallAtK( relevance: ArrayLike<number>, k: number, options: { totalRelevant?: number } = {}, ): number

Re-exported from ranking.recallAtK.

function reciprocalRank(relevance: ArrayLike<number>): number

Re-exported from ranking.reciprocalRank.

function explainedVariance(yTrue: ArrayLike<number>, yPred: ArrayLike<number>): number

Re-exported from regression.explainedVariance.

function maxError(yTrue: ArrayLike<number>, yPred: ArrayLike<number>): number

Re-exported from regression.maxError.

function meanAbsoluteError(yTrue: ArrayLike<number>, yPred: ArrayLike<number>): number

Re-exported from regression.meanAbsoluteError.

function meanAbsolutePercentageError( yTrue: ArrayLike<number>, yPred: ArrayLike<number>, ): number

Re-exported from regression.meanAbsolutePercentageError.

function meanSquaredError(yTrue: ArrayLike<number>, yPred: ArrayLike<number>): number

Re-exported from regression.meanSquaredError.

function meanSquaredLogError(yTrue: ArrayLike<number>, yPred: ArrayLike<number>): number

Re-exported from regression.meanSquaredLogError.

function medianAbsoluteError(yTrue: ArrayLike<number>, yPred: ArrayLike<number>): number

Re-exported from regression.medianAbsoluteError.

function pearsonCorrelation(a: ArrayLike<number>, b: ArrayLike<number>): number

Re-exported from regression.pearsonCorrelation.

function r2Score(yTrue: ArrayLike<number>, yPred: ArrayLike<number>): number

Re-exported from regression.r2Score.

function rootMeanSquaredError(yTrue: ArrayLike<number>, yPred: ArrayLike<number>): number

Re-exported from regression.rootMeanSquaredError.

function cosineDistance(a: ArrayLike<number>, b: ArrayLike<number>): number

Re-exported from similarity.cosineDistance.

function cosineSimilarity(a: ArrayLike<number>, b: ArrayLike<number>): number

Re-exported from similarity.cosineSimilarity.

function dotProduct(a: ArrayLike<number>, b: ArrayLike<number>): number

Re-exported from similarity.dotProduct.

function euclideanDistance(a: ArrayLike<number>, b: ArrayLike<number>): number

Re-exported from similarity.euclideanDistance.

function l2Norm(vector: ArrayLike<number>): number

Re-exported from similarity.l2Norm.

function manhattanDistance(a: ArrayLike<number>, b: ArrayLike<number>): number

Re-exported from similarity.manhattanDistance.

Types

type Average = 'binary' | 'macro' | 'micro' | 'weighted' | 'none'

Re-exported from classification.Average.

type BinStrategy = 'uniform' | 'quantile'

Re-exported from calibration.BinStrategy.

type GainFunction = 'linear' | 'exponential'

Re-exported from ranking.GainFunction.

type Label = string | number | boolean

Re-exported from js/ml/metrics/shared.Label.

Interfaces

interface AverageOptions {

Re-exported from classification.AverageOptions.

interface PrecisionRecallCurve {

Re-exported from classification.PrecisionRecallCurve.

interface ProbabilityOptions {

Re-exported from classification.ProbabilityOptions.

interface RocCurve {

Re-exported from classification.RocCurve.

interface CalibrationBin {

Re-exported from calibration.CalibrationBin.

interface CalibrationOptions {

Re-exported from calibration.CalibrationOptions.

interface GainOptions {

Re-exported from ranking.GainOptions.

interface StreamingMetric<Value> {

Re-exported from streaming.StreamingMetric.

interface StreamingRegressionValue {

Re-exported from streaming.StreamingRegressionValue.

Classes

class ConfusionMatrix {

Re-exported from confusion.ConfusionMatrix.

class StreamingAccuracy implements StreamingMetric<number> {

Re-exported from streaming.StreamingAccuracy.

class StreamingConfusionMatrix implements StreamingMetric<ConfusionMatrix> {

Re-exported from streaming.StreamingConfusionMatrix.

class StreamingMean implements StreamingMetric<number> {

Re-exported from streaming.StreamingMean.

class StreamingRegression implements StreamingMetric<StreamingRegressionValue> {

Re-exported from streaming.StreamingRegression.

class StreamingVariance implements StreamingMetric<number> {

Re-exported from streaming.StreamingVariance.

class MetricError extends Error {

Re-exported from js/ml/metrics/shared.MetricError.