fraudtwin.ml.baseline

Baseline models and prediction evaluation.

Status: Stable

Classes

fraudtwin.ml.baseline.BaselineEvaluationConfig	Strict, analysis-only configuration for M19.
fraudtwin.ml.baseline.EvaluationResult	EvaluationResult(predictions: tuple[dict[str, typing.Any], ...], metrics: tuple[dict[str, typing.Any], ...], manifest: dict[str, typing.Any], model_artifacts: dict[str, bytes] | None = None)
fraudtwin.ml.baseline.PredictionAdapter	Small file/library adapter for external point-in-time predictions.
fraudtwin.ml.baseline.PredictionRecord	One external prediction at a point in time.

Functions

fraudtwin.ml.baseline.evaluate_predictions	Evaluate external scores against a leakage-safe dataset.
fraudtwin.ml.baseline.heuristic_predictions	Score PIT rows with the dependency-free deterministic baseline.
fraudtwin.ml.baseline.load_baseline_config	Load and validate a baseline-model YAML policy.
fraudtwin.ml.baseline.load_model_artifact	Load and validate a persisted baseline artifact once for reuse.
fraudtwin.ml.baseline.load_predictions	Load strict Parquet or JSONL predictions.
fraudtwin.ml.baseline.score_loaded_model	Score rows with an already-loaded baseline artifact.
fraudtwin.ml.baseline.score_model_artifact	Score feature rows with a persisted FraudTwin baseline artifact.
fraudtwin.ml.baseline.train_baselines	Train configured deterministic baselines on train rows only.
fraudtwin.ml.baseline.write_evaluation	Persist predictions, metrics, model artifacts, and the evaluation manifest.
fraudtwin.ml.baseline.write_predictions	Write canonical external predictions as a typed Parquet file.

Constants and protocols

Name	Reference
MODEL_FEATURES	fraudtwin.ml.baseline.MODEL_FEATURES
MODEL_NAMES	fraudtwin.ml.baseline.MODEL_NAMES
PREDICTION_SCHEMA	fraudtwin.ml.baseline.PREDICTION_SCHEMA

Detailed API

Deterministic baseline models and prediction evaluation.

The module intentionally keeps model dependencies lazy. Dataset construction and generation remain usable without the optional ML stack, while a trained run is fully described by local, content-addressed artifacts.

class fraudtwin.ml.baseline.PredictionRecord(**data)

Bases: BaseModel

One external prediction at a point in time.

Parameters:

• event_id (Annotated[str | None, MinLen(min_length=1)])

• payment_id (Annotated[str | None, MinLen(min_length=1)])

• customer_id (Annotated[str | None, MinLen(min_length=1)])

• account_id (Annotated[str | None, MinLen(min_length=1)])

• prediction_timestamp (datetime)

• fraud_score (Annotated[float, Ge(ge=0), Le(le=1)])

• predicted_class (Literal['FRAUD', 'LEGITIMATE'] | None)

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

property score: float

Backward-compatible alias for fraud_score.

fraud_score is the canonical serialized field. The short alias is intentionally read-only so existing scoring notebooks remain concise without creating a second source of truth in prediction artifacts.

class fraudtwin.ml.baseline.PredictionAdapter

Bases: object

Small file/library adapter for external point-in-time predictions.

class fraudtwin.ml.baseline.BaselineEvaluationConfig(**data)

Bases: BaseModel

Strict, analysis-only configuration for M19.

Parameters:

• models (tuple[str, ...])

• fixed_fpr (Annotated[float, Gt(gt=0), Lt(lt=1)])

• fixed_recall (Annotated[float, Gt(gt=0), Lt(lt=1)])

• ranking_k_fraction (Annotated[float, Gt(gt=0), Le(le=1)])

• label_policy (Literal['exclude_unresolved'])

• random_seed (Annotated[int | None, Ge(ge=0)])

• deterministic_presets (dict[str, Any])

• tracking_uri (str | None)

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class fraudtwin.ml.baseline.EvaluationResult(predictions, metrics, manifest, model_artifacts=None)

Bases: object

Parameters:

• predictions (tuple[dict[str, Any], ...])

• metrics (tuple[dict[str, Any], ...])

• manifest (dict[str, Any])

• model_artifacts (dict[str, bytes] | None)

fraudtwin.ml.baseline.load_baseline_config(path)

Load and validate a baseline-model YAML policy.

Parameters:

path (Path) – YAML file containing model names, thresholds, and tracking options.

Return type:

BaselineEvaluationConfig

Returns:

An immutable BaselineEvaluationConfig.

Raises:

• FileNotFoundError – If path does not exist.

• ValueError – If the YAML root or any option is invalid.

fraudtwin.ml.baseline.load_predictions(path)

Load strict Parquet or JSONL predictions.

Return type:

tuple[PredictionRecord, ...]

Parameters:

path (Path)

fraudtwin.ml.baseline.write_predictions(predictions, path)

Write canonical external predictions as a typed Parquet file.

Parameters:

• predictions (Iterable[PredictionRecord]) – Point-in-time scores keyed by exactly one target ID.

• path (Path) – Destination path; parent directories are created as needed.

Return type:

Path

Returns:

The destination path.

fraudtwin.ml.baseline.score_model_artifact(path, rows)

Score feature rows with a persisted FraudTwin baseline artifact.

Parameters:

• path (Path) – .joblib artifact written under an evaluation models directory.

• rows (Sequence[Mapping[str, Any]]) – PIT-shaped feature mappings. Missing numeric values use the same deterministic zero policy used during training.

Return type:

tuple[float, ...]

Returns:

Fraud probabilities in input order.

Raises:

• FileNotFoundError – If the artifact path is missing.

• RuntimeError – If joblib is not installed.

• ValueError – If the artifact does not contain a compatible model.

fraudtwin.ml.baseline.load_model_artifact(path)

Load and validate a persisted baseline artifact once for reuse.

Return type:

dict[str, Any]

Parameters:

path (Path)

fraudtwin.ml.baseline.score_loaded_model(artifact, rows)

Score rows with an already-loaded baseline artifact.

Return type:

tuple[float, ...]

Parameters:

• artifact (Mapping[str, Any])

• rows (Sequence[Mapping[str, Any]])

fraudtwin.ml.baseline.evaluate_predictions(rows, predictions, config, *, model_id='external', source_run_dir=None)

Evaluate external scores against a leakage-safe dataset.

Parameters:

• rows (Sequence[Mapping[str, Any]]) – PIT rows containing split and label columns.

• predictions (Sequence[PredictionRecord]) – Scores keyed by the row’s event, payment, customer, or account ID.

• config (BaselineEvaluationConfig) – Threshold and metric policy.

• model_id (str) – Stable name recorded in metrics and lineage.

• source_run_dir (Path | None) – Optional run directory used to enrich lineage metadata.

Return type:

EvaluationResult

Returns:

Predictions, partition/segment metrics, and a reproducibility manifest.

Raises:

ValueError – If predictions do not resolve one-to-one to PIT rows.

fraudtwin.ml.baseline.heuristic_predictions(rows)

Score PIT rows with the dependency-free deterministic baseline.

Return type:

tuple[PredictionRecord, ...]

Parameters:

rows (Sequence[Mapping[str, Any]])

fraudtwin.ml.baseline.train_baselines(rows, config, *, source_run_dir=None)

Train configured deterministic baselines on train rows only.

Parameters:

• rows (Sequence[Mapping[str, Any]]) – PIT rows with train, validation, and test splits.

• config (BaselineEvaluationConfig) – Model list and evaluation policy.

• source_run_dir (Path | None) – Optional source run used for lineage metadata.

Return type:

EvaluationResult

Returns:

Evaluation results and in-memory model artifacts for optional models.

Raises:

• ValueError – If train/validation partitions or both classes are missing.

• RuntimeError – If an optional model dependency is unavailable.

fraudtwin.ml.baseline.write_evaluation(result, output_dir)

Persist predictions, metrics, model artifacts, and the evaluation manifest.

Parameters:

• result (EvaluationResult) – Output returned by train_baselines() or evaluate_predictions().

• output_dir (Path) – New directory for the immutable evaluation artifacts.

Return type:

tuple[Path, Path, Path]

Returns:

Paths to predictions, metrics, and manifest files.

Raises:

• FileExistsError – If output_dir already exists.

• RuntimeError – If MLflow tracking is requested but not installed.